From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on yquem.inria.fr X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=AWL autolearn=disabled version=3.1.3 Received: from mail3-relais-sop.national.inria.fr (mail3-relais-sop.national.inria.fr [192.134.164.104]) by yquem.inria.fr (Postfix) with ESMTP id D69B3BC69 for ; Wed, 14 Nov 2007 17:36:18 +0100 (CET) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAADazOkfAXQInh2dsb2JhbACPAwIBCAopgQ8 X-IronPort-AV: E=Sophos;i="4.21,417,1188770400"; d="scan'208";a="5803722" Received: from concorde.inria.fr ([192.93.2.39]) by mail3-smtp-sop.national.inria.fr with ESMTP; 14 Nov 2007 17:36:18 +0100 Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by concorde.inria.fr (8.13.6/8.13.6) with ESMTP id lAEGaICQ005995 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=OK) for ; Wed, 14 Nov 2007 17:36:18 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ao8CAAqyOkdCm3xr/2dsb2JhbACQUA X-IronPort-AV: E=Sophos;i="4.21,417,1188770400"; d="scan'208";a="4240889" Received: from janestcapital.com (HELO smtp.janestcapital.com) ([66.155.124.107]) by mail2-smtp-roc.national.inria.fr with ESMTP; 14 Nov 2007 17:36:17 +0100 Received: from [172.25.129.161] [38.96.172.125] by janestcapital.com with ESMTP (SMTPD-9.10) id A400091C; Wed, 14 Nov 2007 11:36:16 -0500 Message-ID: <473B2400.5050800@janestcapital.com> Date: Wed, 14 Nov 2007 11:36:16 -0500 From: Brian Hurt User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Stefan Monnier Cc: caml-list@inria.fr Subject: Re: [Caml-list] Re: OCaml runtime using too much memory in 64-bit Linux References: <4731F5D1.2070405@janestcapital.com> <1194459622.15728.94.camel@localhost.localdomain> <47320E10.1050307@janestcapital.com> <200711140520.46016.romain.beauxis@gmail.com> <8ef825670711140403m464c01detff5537abb2211946@mail.gmail.com> <473AF04C.7030107@inria.fr> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Miltered: at concorde with ID 473B2402.000 by Joe's j-chkmail (http://j-chkmail . ensmp . fr)! X-Spam: no; 0.00; ocaml:01 runtime:01 allocating:01 initialized:01 allocates:01 byte:01 allocations:01 segfault:01 wrote:01 caml-list:01 allocated:02 allocated:02 bytes:03 brian:05 brian:05 Stefan Monnier wrote: >How about allocating this array of bytes via mmap and then leave it >uninitialized (relying on POSIX's guarantee that it's already >initialized to zeros)? >This way you can easily have a 4GB "dense" table which doesn't use much >RAM since most of the 4GB will be mapped (via copy-on-write) to the same >"zero page". > > > Even on a system like linux, which optimistically allocates memory (i.e. the actually underlying memory isn't allocated until you actually touch it), once you read the page, it has to actually exist in memory. Even using 1 byte per 4M page, mapping a whole 64-bit memory space requires 4 TB of ram. And many systems do not optimistically allocate memory, as it causes a lot of problems (for example, allocations can return false positives, which then segfault when you first touch the memory because it can't really be allocated). Brian