OCaml, C, Garbage Collector etc

OCaml, C, Garbage Collector etc

[Odalric-Ambrym Maillard]

[-- Attachment #1: Type: text/plain, Size: 5585 bytes --]

Hello,

I am currently writin an algorithm in OCaml. Since its a learning algorithm,
I would like to uses some existing code, which is mainly in C.
For the program to be usable by others, I write the main structure in C and
make call to some functions in Ocaml when needed.

Well, I thought I had understood how this works. but obviously, there is
omething wrong, for the program crashes, and it seems, according to my tests
and sylvain le-gall, that it is due to problem concerning the garbage
collector.

All I need is this:
- say I have an OCaml  function psi : int -> float array -> float.
- say  have a C matrix of data:  double ** x_data, and a vector double *
y_data.

Then, I just need to create, in C a matrix, double ** psi_mat, with values
psi m x_i at position m,i, where w_i is the ith row of x_data.
One way to do that (not the most efficient) is to create the function psi_c
corresponding to the OCaml psi function, and then use it in the code.

For instance, let consider this code:
--------------------------------------------------------------------
double psi_c(int m, int dim_x,double * x){
  int d;

  CAMLlocal2(val_M,val_x);
  val_m = Val_int(m);
  val_x = caml_alloc(dim_x * Double_wosize, Double_array_tag);
  for(d=0;d<dim_x;d++){
    printf("Stored %f",x[d]);
    Store_double_field(val_x, d,x[d]);
  }

  return(Double_val(caml_callback2( *closure_psi, val_m, val_x)));
  }
--------------------------------------------------------------------
where of course, the following variable is declared globally:

static value * closure_psi = NULL;

and  somewhere in the main we have:

 if (closure_psi == NULL) {
    closure_psi = (value*) caml_named_value("psi_Ocaml");
  }


Well, this code does not work, meaning that a call to this function may
generate either segmentation fault, or infinite loop, or just the correct
expected result. This impredictible behaviour seems to come from the Garbage
collector that moves blocks it should not move here...

But generally, you can handle this when you write a function C from OCaml,
since you have a function of the form:

value my_function(value x){
CAMLparam1(x)
...
CAMLreturn(...)
}
thus CAMLparam and CAMLreturn seem to do the job.

But how to do that when you are on the other side ??

I really do not understand hoxw to do that. I suspect moreover that there is
something specific to be done due to the fact I use double values, which are
handled in a specific way by OCaml (so may be the same code would work with
int values (or not ??) ? anyway I need double).




Eventually, what I would prefer is to do only one allocation of the ith row
x_i to call the corresponding OCaml, like the following code (which works
when executed !?!) :


------------------------------------------------------------------------------------------------
void make_psi_matrix(int Mnumber_of_compressed_features,
             int Knumber_of_training_data,
             int Dspace_dimension,
             double **x_train,
             double ***psi_matrix) {

  double * xk;
  double ** psi_matrixm;
  int m,d,k;


    xk = malloc (Dspace_dimension * sizeof (double));
  *psi_matrix = malloc (Mnumber_of_compressed_features * sizeof (double*));
  for(m=0,psi_matrixm = *psi_matrix;
      m<Mnumber_of_compressed_features;
      m++,psi_matrixm++) {
    *psi_matrixm = malloc  (Knumber_of_training_data * sizeof (double));
  }

  CAMLlocal2(val_x,val_M);
  val_x = caml_alloc(Dspace_dimension * Double_wosize, Double_array_tag);
  for(k=0;k<Knumber_of_training_data;k++){
    xk = x_train[k];
    //printf("k :%d/%d\n",k,Knumber_of_training_data);
    for(d=0;d<Dspace_dimension;d++){
      //printf("Store %f at position %d,%d...",xk[d],k,d);
      Store_double_field(val_x, d, xk[d]);
      // printf("...ok\n");
    }

    for(m=0;m<Mnumber_of_compressed_features;m++){

      val_M = Val_int(m);
      ((*psi_matrix)[m])[k] = Double_val(caml_callback2( *closure_psi,
val_M, val_x));

    }
    }


}


------------------------------------------------------------------------------------------------
What is very suprisin is that its brother that outputs a vector y_predict of
prediciton does not work at all (though the code  is very similar for the
Ocaml call).
Thus we have the same problem again : how to handle  this more complicated
case ? Suppose I can use CAMLparam in the following code : How can I realluy
use it ?? I really do not know because of the loops.

------------------------------------------------------------------------------------------------
void predict(double * beta,
         int Mnumber_of_compressed_features,
         int Dspace_dimension,
         int Knumber_of_data,
         double **x_data,
         double ** y_predict) {


  int d,m,k;
  double s,aux_val;
  double *xk;

  xk = malloc (Dspace_dimension * sizeof (double));
  *y_predict = malloc (Knumber_of_data * sizeof (double));
  CAMLlocal2(val_x,val_M);
  val_x = caml_alloc(Dspace_dimension * Double_wosize, Double_array_tag);
  for(k=0;k<Knumber_of_data;k++){
    xk = x_data[k];
    for(d=0;d<Dspace_dimension;d++){
      Store_double_field(val_x, d, xk[d]);
    }
    s = 0;
    for(m=0;m<Mnumber_of_compressed_features;m++){
      val_M = Val_int(m);
      s = s + beta[m]*Double_val(caml_callback2( *closure_psi, val_M,
val_x));
    }
    (*y_predict)[k] = s;
  }
}

------------------------------------------------------------------------------------------------


So, does anybody has a solution, idea, or even a good example (using vectors
or matrix of double, I insist) that could help ?

Have  a good day,
Odalric-Ambrym

[-- Attachment #2: Type: text/html, Size: 6326 bytes --]

Re: OCaml, C, Garbage Collector etc

[Odalric-Ambrym Maillard]

[-- Attachment #1: Type: text/plain, Size: 2474 bytes --]

Hello,

I have reduced my problem to the following question :

I have this code in C, these are the wrappers corresponding to the Ocaml
functions. I call the three following functions from C code. They correspond
respectively to functions of type : int -> unit,  int -> unit and int ->
float array -> float.
*
My concern is about the good (w.r.t gc) way of calling Ocaml functions with
return type unit from C.*

Thus where to put Camlparam, and Camlreturn and how etc... I have tried this
but it fails too :
------------------------------------------------------------
void init_tree_c(int m) {
  CAMLparam0();
  CAMLlocal2(val_m,res);
  val_m = Val_int(m);
  res=caml_callback(*closure_init_tree, val_m);
  CAMLreturn0;
}

void set_depth_c(int max_depth) {
  CAMLparam0();
  CAMLlocal2(val_max_depth,res);
  val_max_depth = Val_int(max_depth);
  res=caml_callback(*closure_set_depth, val_max_depth);
  CAMLreturn0;
}

double psi_c(int m, int dim_x,double * x){
  int d;
  CAMLparam0();
  CAMLlocal3(val_m,val_x,res);
  val_m = Val_int(m);
  val_x = caml_alloc(dim_x * Double_wosize, Double_array_tag);
  for(d=0;d<dim_x;d++){
    //    printf("Stored %f",x[d]);
    Store_double_field(val_x, d,x[d]);
  }
  res = caml_callback2( *closure_psi, val_m, val_x);
  CAMLreturn(Double_val(res));
  }
------------------------------------------------------------

Here is the corresponding OCaml Code (with an additionnal get_tree
function):
(I have added some printers for debug purpose).

------------------------------------------------------------
let (init_tree, get_tree, set_depth, psi) =
  let random_tree = ref [|Empty|] and depth = ref 3 in
    (
      (function m -> random_tree := Array.make m Empty;print_int
m;print_string " trees initialized.";print_newline();),
      (function m -> Array.get !random_tree m;),
      (function d -> depth := d;print_string "Maximum depth initialized to
";print_int d;print_newline();),
      (function m -> (function x ->
            let t,v = aux (Array.get !random_tree m) !depth 0 1
(Array.length x) x in
            Array.set !random_tree m t;
            v))
    );;
------------------------------------------------------------
Note that : random_tree contains an arrray of m trees, and that this
variable is shared by the different functions.

Of course there is the register code  etc...

Callback.register "init_tree_Ocaml" init_tree;;
Callback.register "set_depth_Ocaml" set_depth;;
Callback.register "psi_Ocaml" psi;;
...

[-- Attachment #2: Type: text/html, Size: 2800 bytes --]