[Caml-list] Conditionally boxed 32 bit integers?

[Caml-list] Conditionally boxed 32 bit integers?

[John Carr]

I am working with a file format the contains 32 bit integers.
I need to use int32 on 32 bit systems.  I would like to use plain
integers, unboxed and with native machine operations, on 64 bit
systems.

Is there any way to convince ocamlopt to choose between int and
int32 representations _at compile time_?

I could use first class modules to select implementations at
runtime, which is not worth the code complexity and still
requires an indirect function call to add numbers.  I could
conditionally compile different files depending on word size,
which strikes me as an ugly and fragile solution.

I want to be able to write "int32" and have that be compiled
like an ordinary integer type if 32 bits and tag fit into a
word, and as a boxed type otherwise.  (The compiler could
mask off excess precision if desired.)  Is there a reason ocaml
can't provide this?

Another implementation would have each instance of an int32 or
int64 be boxed or not depending on whether the value fits into
a word.  I don't know whether this would be faster or slower in
practice.  There is a tradeoff between allocations and
conditional branches.

    --John Carr (jfc@mit.edu)

Re: [Caml-list] Conditionally boxed 32 bit integers?

[Hezekiah M. Carty]

On Sun, Sep 4, 2011 at 11:22 AM, John Carr <jfc@mit.edu> wrote:
>
> I am working with a file format the contains 32 bit integers.
> I need to use int32 on 32 bit systems.  I would like to use plain
> integers, unboxed and with native machine operations, on 64 bit
> systems.
>
> Is there any way to convince ocamlopt to choose between int and
> int32 representations _at compile time_?
>

You may be able to use ideas/implementation from the Int63 module:

http://people.redhat.com/~rjones/virt-df/html/Int63.html
or
http://www.janestreet.com/ocaml/janestreet-ocamldocs/core/Int63.html

The Int63 module uses int internally on 64bit OCaml installations and
Int64.t internally on 32bit OCaml installations.  I am not familiar
with the details but it may be adaptable to your needs.

Hez

Re: [Caml-list] Conditionally boxed 32 bit integers?

[Richard W.M. Jones]

On Sun, Sep 04, 2011 at 11:22:05AM -0400, John Carr wrote:
> 
> I am working with a file format the contains 32 bit integers.
> I need to use int32 on 32 bit systems.  I would like to use plain
> integers, unboxed and with native machine operations, on 64 bit
> systems.
> 
> Is there any way to convince ocamlopt to choose between int and
> int32 representations _at compile time_?

The Bitstring module could use this too ...

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] Conditionally boxed 32 bit integers?

[rixed]

I suppose you don't want to use any kind of preprocessor, do you ?

Re: [Caml-list] Conditionally boxed 32 bit integers?

[John Carr]

rixed@happyleptic.org wrote:

> I suppose you don't want to use any kind of preprocessor, do you ?

I'd rather not preprocess (or compile alternate files).  Both the
netnumber and core implementations use a preprocessor.

Preprocessor-based solutions have two problems.  One is the extra
compilation step.  Another is bitrot in the unused case.

I think it was Plan 9 where the documentation said don't use the
C preprocessor, our compiler does the right thing with if (false).
In ocaml what I'd like to say is

	module Int : sig
	  type t
	  val add : t -> t -> t
	  ...
	end;
	module Intfast : Int = struct type t = int ... end
	module Int = if sixtyfour then Int end else Int32

with the understanding that the expression sixtyfour is evaluated
at compile time.

Here is some similar code using 3.12 syntax:

	module type M = sig type t val add : t -> t -> t val of_int : int -> t end;;
	module Int : M = struct type t = int let add = (+) let of_int x = x end;;
	module M = (val (if Sys.word_size = 64 then (module Int : M) else (module Int32 : M)) : M);;

	let f x = M.add (M.of_int x) (M.of_int 1)

If the test (Sys.word_size = 64) is replaced by (true) this does what
I want.  The compiler looks through the packing and unpacking and
sees the module Int.  The body of function f compiles to a machine add
instruction.

Ocamlopt does not evaluate (Sys.word_size = 64) at compile time
because Sys.word_size is initialized at runtime.  See sys.ml line 26
and run objinfo on sys.cmx.

INRIA developers, is it easy to add an intrinsic so we can write

	extern word_size : int = "%caml_word_size"

in sys.ml?  If sys.cmx has a constant definition, ocamlopt should do
constant folding on conditional expressions testing Sys.word_size.

If I want to do conditional compilation, can ocamlbuild be taught
"compile file a.ml as module X on 32 bit systems and module b.ml as
module X on 64 bit systems"?

    --John Carr (jfc@mit.edu)

Re: [Caml-list] Conditionally boxed 32 bit integers?

[John Carr]

I found a simpler way to make Sys.word_size be a compile time constant.
See patch at end.  Technically this is preprocessing, which I wanted to
avoid, but it's not preprocessing in my code.

> INRIA developers, is it easy to add an intrinsic so we can write
> 
> 	extern word_size : int = "%caml_word_size"
> 
> in sys.ml?  If sys.cmx has a constant definition, ocamlopt should do
> constant folding on conditional expressions testing Sys.word_size.


diff -rc /tmp/ocaml-3.12.1/stdlib/Makefile.shared ./stdlib/Makefile.shared
*** /tmp/ocaml-3.12.1/stdlib/Makefile.shared	Fri May 21 07:28:21 2010
--- ./stdlib/Makefile.shared	Mon Sep  5 17:39:14 2011
***************
*** 49,56 ****
  stdlib.cmxa: $(OBJS:.cmo=.cmx)
  	$(CAMLOPT) -a -o stdlib.cmxa $(OBJS:.cmo=.cmx)
  
! sys.ml: sys.mlp ../VERSION
! 	sed -e "s|%%VERSION%%|`sed -e 1q ../VERSION`|" sys.mlp >sys.ml
  
  clean::
  	rm -f sys.ml
--- 49,57 ----
  stdlib.cmxa: $(OBJS:.cmo=.cmx)
  	$(CAMLOPT) -a -o stdlib.cmxa $(OBJS:.cmo=.cmx)
  
! sys.ml: sys.mlp ../VERSION ../config/m.h
! 	ws=`sed -n -e 's/^#define ARCH_SIXTYFOUR/64/p' -e 's/^#undef ARCH_SIXTYFOUR/32/p' ../config/m.h`; \
! 	sed -e "s|%%VERSION%%|`sed -e 1q ../VERSION`|" -e "s|%%WORD_SIZE%%|$$ws|" sys.mlp >sys.ml
  
  clean::
  	rm -f sys.ml
diff -rc /tmp/ocaml-3.12.1/stdlib/sys.mlp ./stdlib/sys.mlp
*** /tmp/ocaml-3.12.1/stdlib/sys.mlp	Mon Feb 26 09:21:57 2007
--- ./stdlib/sys.mlp	Mon Sep  5 17:35:33 2011
***************
*** 23,29 ****
  external get_argv: unit -> string * string array = "caml_sys_get_argv"
  
  let (executable_name, argv) = get_argv()
! let (os_type, word_size) = get_config()
  let max_array_length = (1 lsl (word_size - 10)) - 1;;
  let max_string_length = word_size / 8 * max_array_length - 1;;
  
--- 23,30 ----
  external get_argv: unit -> string * string array = "caml_sys_get_argv"
  
  let (executable_name, argv) = get_argv()
! let (os_type, _) = get_config()
! let word_size = %%WORD_SIZE%%;;
  let max_array_length = (1 lsl (word_size - 10)) - 1;;
  let max_string_length = word_size / 8 * max_array_length - 1;;
  

Re: [Caml-list] Conditionally boxed 32 bit integers?

[Fabrice Le Fessant]

[-- Attachment #1: Type: text/plain, Size: 2583 bytes --]

I think the (old) reason of the way Sys.word_size is implemented is that
the same code can run both on 32 bits and 64 bits, when compiled to
bytecode. But of course, I see no reason why such a primitive could not
be added, with different compilation schemes for asm (it would be hard
coded) and bytecode (it would continue to call the runtime function).

Fabrice


On 09/06/2011 12:11 AM, John Carr wrote:
> I found a simpler way to make Sys.word_size be a compile time constant.
> See patch at end.  Technically this is preprocessing, which I wanted to
> avoid, but it's not preprocessing in my code.
> 
>> INRIA developers, is it easy to add an intrinsic so we can write
>>
>> 	extern word_size : int = "%caml_word_size"
>>
>> in sys.ml?  If sys.cmx has a constant definition, ocamlopt should do
>> constant folding on conditional expressions testing Sys.word_size.
> 
> 
> diff -rc /tmp/ocaml-3.12.1/stdlib/Makefile.shared ./stdlib/Makefile.shared
> *** /tmp/ocaml-3.12.1/stdlib/Makefile.shared	Fri May 21 07:28:21 2010
> --- ./stdlib/Makefile.shared	Mon Sep  5 17:39:14 2011
> ***************
> *** 49,56 ****
>   stdlib.cmxa: $(OBJS:.cmo=.cmx)
>   	$(CAMLOPT) -a -o stdlib.cmxa $(OBJS:.cmo=.cmx)
>   
> ! sys.ml: sys.mlp ../VERSION
> ! 	sed -e "s|%%VERSION%%|`sed -e 1q ../VERSION`|" sys.mlp >sys.ml
>   
>   clean::
>   	rm -f sys.ml
> --- 49,57 ----
>   stdlib.cmxa: $(OBJS:.cmo=.cmx)
>   	$(CAMLOPT) -a -o stdlib.cmxa $(OBJS:.cmo=.cmx)
>   
> ! sys.ml: sys.mlp ../VERSION ../config/m.h
> ! 	ws=`sed -n -e 's/^#define ARCH_SIXTYFOUR/64/p' -e 's/^#undef ARCH_SIXTYFOUR/32/p' ../config/m.h`; \
> ! 	sed -e "s|%%VERSION%%|`sed -e 1q ../VERSION`|" -e "s|%%WORD_SIZE%%|$$ws|" sys.mlp >sys.ml
>   
>   clean::
>   	rm -f sys.ml
> diff -rc /tmp/ocaml-3.12.1/stdlib/sys.mlp ./stdlib/sys.mlp
> *** /tmp/ocaml-3.12.1/stdlib/sys.mlp	Mon Feb 26 09:21:57 2007
> --- ./stdlib/sys.mlp	Mon Sep  5 17:35:33 2011
> ***************
> *** 23,29 ****
>   external get_argv: unit -> string * string array = "caml_sys_get_argv"
>   
>   let (executable_name, argv) = get_argv()
> ! let (os_type, word_size) = get_config()
>   let max_array_length = (1 lsl (word_size - 10)) - 1;;
>   let max_string_length = word_size / 8 * max_array_length - 1;;
>   
> --- 23,30 ----
>   external get_argv: unit -> string * string array = "caml_sys_get_argv"
>   
>   let (executable_name, argv) = get_argv()
> ! let (os_type, _) = get_config()
> ! let word_size = %%WORD_SIZE%%;;
>   let max_array_length = (1 lsl (word_size - 10)) - 1;;
>   let max_string_length = word_size / 8 * max_array_length - 1;;
>   
> 

[-- Attachment #2: fabrice_le_fessant.vcf --]
[-- Type: text/x-vcard, Size: 380 bytes --]

begin:vcard
fn:Fabrice LE FESSANT
n:LE FESSANT;Fabrice
org:INRIA Saclay -- Ile-de-France;P2P & OCaml
adr;quoted-printable:;;Parc Orsay Universit=C3=A9 ;Orsay CEDEX;;91893;France
email;internet:fabrice.le_fessant@inria.fr
title;quoted-printable:Charg=C3=A9 de Recherche
tel;work:+33 1 74 85 42 14
tel;fax:+33 1 74 85 42 49 
url:http://fabrice.lefessant.net/
version:2.1
end:vcard

Re: [Caml-list] Conditionally boxed 32 bit integers?

[Gerd Stolpmann]

Am Sonntag, den 04.09.2011, 11:22 -0400 schrieb John Carr:
> I am working with a file format the contains 32 bit integers.
> I need to use int32 on 32 bit systems.  I would like to use plain
> integers, unboxed and with native machine operations, on 64 bit
> systems.
> 
> Is there any way to convince ocamlopt to choose between int and
> int32 representations _at compile time_?

The Netnumber module of Ocamlnet actually does this:

https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netstring/netnumber.mli
https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netstring/netnumber.mlp

Netnumber is fairly standalone, so you could just copy it to your
project if you don't want all of Ocamlnet.

Gerd


> 
> I could use first class modules to select implementations at
> runtime, which is not worth the code complexity and still
> requires an indirect function call to add numbers.  I could
> conditionally compile different files depending on word size,
> which strikes me as an ugly and fragile solution.
> 
> I want to be able to write "int32" and have that be compiled
> like an ordinary integer type if 32 bits and tag fit into a
> word, and as a boxed type otherwise.  (The compiler could
> mask off excess precision if desired.)  Is there a reason ocaml
> can't provide this?
> 
> Another implementation would have each instance of an int32 or
> int64 be boxed or not depending on whether the value fits into
> a word.  I don't know whether this would be faster or slower in
> practice.  There is a tradeoff between allocations and
> conditional branches.
> 
>     --John Carr (jfc@mit.edu)
> 

-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd@gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
*** Searching for new projects! Need consulting for system
*** programming in Ocaml? Gerd Stolpmann can help you.
------------------------------------------------------------