From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.1 required=5.0 tests=AWL autolearn=disabled version=3.1.3 Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by yquem.inria.fr (Postfix) with ESMTP id DED6FBC6B for ; Wed, 7 Nov 2007 19:20:34 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAABeRMUfAXQInh2dsb2JhbACBW40iAgEICik X-IronPort-AV: E=Sophos;i="4.21,385,1188770400"; d="scan'208";a="4001296" Received: from concorde.inria.fr ([192.93.2.39]) by mail2-smtp-roc.national.inria.fr with ESMTP; 07 Nov 2007 19:20:34 +0100 Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id lA7IKTe2007556 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=OK) for ; Wed, 7 Nov 2007 19:20:34 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAABeRMUfU4366nmdsb2JhbACBW40iAgEBBwIIERg X-IronPort-AV: E=Sophos;i="4.21,385,1188770400"; d="scan'208";a="5556012" Received: from moutng.kundenserver.de ([212.227.126.186]) by mail3-smtp-sop.national.inria.fr with ESMTP; 07 Nov 2007 19:20:25 +0100 Received: from gate.lan.gerd-stolpmann.de (dslb-084-059-100-181.pools.arcor-ip.net [84.59.100.181]) by mrelayeu.kundenserver.de (node=mrelayeu3) with ESMTP (Nemesis) id 0MKxQS-1IppVn0A9i-0006g4; Wed, 07 Nov 2007 19:20:24 +0100 Received: from flakew.lan.gerd-stolpmann.de (fw.lan.gerd-stolpmann.de [192.168.1.1]) by gate.lan.gerd-stolpmann.de (Postfix) with ESMTP id 76DBFC0C6; Wed, 7 Nov 2007 19:20:22 +0100 (CET) Subject: Re: [Caml-list] OCaml runtime using too much memory in 64-bit Linux From: Gerd Stolpmann To: Adam Chlipala Cc: caml-list@inria.fr In-Reply-To: <4731F5D1.2070405@janestcapital.com> References: <4731F5D1.2070405@janestcapital.com> Content-Type: text/plain Date: Wed, 07 Nov 2007 19:20:21 +0100 Message-Id: <1194459622.15728.94.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.6.1 Content-Transfer-Encoding: 7bit X-Provags-ID: V01U2FsdGVkX19AEQnbvLSZ/7Gthpe/LO4b3FJFndAJvb0VVPP 7OHbtJZA7qy72h2WKKsNQtmvTogbWTpv3oiAxffrN7I0qouJRa zTeIMBC5zyWgZ3IA4FGU2dBZdMGamqf X-Miltered: at concorde with ID 473201ED.000 by Joe's j-chkmail (http://j-chkmail . ensmp . fr)! X-Spam: no; 0.00; ocaml:01 runtime:01 gerd:01 stolpmann:01 ocaml:01 runtime:01 malloc:01 o'caml:01 gerd:01 stolpmann:01 viktoriastr:01 64293:01 darmstadt:01 6151:01 6151:01 Am Mittwoch, den 07.11.2007, 12:28 -0500 schrieb Adam Chlipala: > I've encountered a problem where certain OCaml programs use orders of > magnitude more RAM when compiled/run in 64-bit Linux instead of 32-bit > Linux. Some investigation led to the conclusion that the difference has > to do with the size of OCaml page tables. (Here I mean the page tables > maintained by the OCaml runtime system, not any OS stuff.) > > A program that should be using just a few megabytes of RAM ends up using > 200+ MB to store a page table. It seems that a C macro is defined by > default on 64-bit Linux to use mmap() instead of malloc(). Ironically, > a comment says that this was done to avoid being given blocks of memory > that are very far apart from each other, forcing the creation of overly > large page tables. It's ironic because that is exactly the problem that > is showing up now with mmap(). It ends up called twice for the program > I'm looking at, and the two addresses it returns are far enough apart to > lead to creation of a 200 MB page table. > > Has anyone else experienced this problem? Would the runtime system need > to be changed to avoid it? We are using O'Caml on 64 bit Linux, and aren't aware of such problems. Did you observe a debug GC message that proves it? 200 MB means that an address space of 200M * 4K = 8E is covered. Also think of Linux modifications that do address randomization, i.e. prevent that contiguous addresses are allocated. Gerd -- ------------------------------------------------------------ Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany gerd@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ------------------------------------------------------------