From mboxrd@z Thu Jan 1 00:00:00 1970 Received: (from weis@localhost) by pauillac.inria.fr (8.7.6/8.7.3) id IAA20340 for caml-redistribution; Tue, 14 Dec 1999 08:51:49 +0100 (MET) Received: from nez-perce.inria.fr (nez-perce.inria.fr [192.93.2.78]) by pauillac.inria.fr (8.7.6/8.7.3) with ESMTP id UAA29799 for ; Mon, 13 Dec 1999 20:16:46 +0100 (MET) Received: from cepheus.azstarnet.com (cepheus.azstarnet.com [169.197.56.195]) by nez-perce.inria.fr (8.8.7/8.8.7) with ESMTP id UAA27853 for ; Mon, 13 Dec 1999 20:16:44 +0100 (MET) Received: from dylan (dialup09ip066.tus.azstarnet.com [169.197.34.66]) by cepheus.azstarnet.com (8.9.3+blt.Beta0/8.9.3) with SMTP id MAA10362; Mon, 13 Dec 1999 12:16:34 -0700 (MST) X-Sent-via: StarNet http://www.azstarnet.com/ Message-ID: <001d01bf459e$cf7d7c00$210148bf@dylan> From: "David McClain" To: , "John Carol Langford" Subject: Re: ocaml limitations Date: Mon, 13 Dec 1999 12:18:09 -0700 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3 Sender: weis I have implemented an extension to OCAML for large numeric arrays. This is a C module (actually C++) and the measurements that I have made for it show only a 10% speed penalty compared to native OCAML arrays. Of course it depends on how you make this measurement. The overhead is confined to an initial exploration of the type of datum, and so it can be amortized quite nicely when you really do have large arrays. My measurements were made by faking the maximum allowable OCAML array size to some small value like 100 elements or so, so that the external extension would get exercised. Under this circumstance the overhead was measured at 10-11%. BTW this code is on the Hump and available from there to my Web site. - David McClain -----Original Message----- From: John Carol Langford To: caml-list@inria.fr Date: Monday, December 13, 1999 8:25 AM Subject: ocaml limitations >I have been encountering some fundamental limitations of the ocaml >language and compiler that are killing my performance - to the tune of >a factor of 10 off equivalent C code. This is a serious problem >because the program I'm working on is both RAM and cpu intensive. > >The performance problems from two limitations. The first is in the >compiler and runtime - it's the limitation on the array size on 32 bit >machines - I only have linux PC's available to work on. It appears >that the garbage collector needs some type information to work with >arrays and enough bits are set aside for type information that not >enough bits are allowed to specify a large array size. You can get >around this large array size problem by simulating a large array with >an array of arrays, but there is a significant performance penalty. > >The second problem is a language failure - there is no 'short int' >type in ocaml. Due to the combinatorics of my problem it would be >very convenient to use 16 bit integers. Using 32 bit integers instead >doubles the footprint of the program - which is unacceptable in this >case. Consequently, I simulate 16 bit integers using masking games - >which again incurs a performance penalty. > >These two problems together add up to using a function considerably >more complicated then an array dereference on the inner loop: > >let get_first i = i land (num_array -1) >let get_second i = i lsr (log_num_array+1) >let get_third i = (i lsr log_num_array) land 1 > >let get_split split i = > let first_index = get_first i > and second_index = get_second i > and third_index = get_third i in > let ret = split.(first_index).(second_index) in > let real_ret = > if third_index = 1 then ret > else ret lsr 16 in > real_ret land 65535 > >Naturally, the performance (relative to what the hardware is capable >of) is terrible. Consequently, I'm wondering if there are plans to >remove either (or both) of these limitations in the near future and >lacking that if there are better workarounds. Any suggestions? > >-John >