From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail2-relais-roc.national.inria.fr (mail2-relais-roc.national.inria.fr [192.134.164.83]) by sympa.inria.fr (Postfix) with ESMTPS id 4C9BB7EE51 for ; Fri, 17 May 2013 18:16:04 +0200 (CEST) Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of rich@annexia.org) identity=pra; client-ip=80.68.91.176; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rich@annexia.org"; x-sender="rich@annexia.org"; x-conformance=sidf_compatible Received-SPF: Pass (mail2-smtp-roc.national.inria.fr: domain of rich@annexia.org designates 80.68.91.176 as permitted sender) identity=mailfrom; client-ip=80.68.91.176; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rich@annexia.org"; x-sender="rich@annexia.org"; x-conformance=sidf_compatible; x-record-type="v=spf1" Received-SPF: None (mail2-smtp-roc.national.inria.fr: no sender authenticity information available from domain of postmaster@furbychan.cocan.org) identity=helo; client-ip=80.68.91.176; receiver=mail2-smtp-roc.national.inria.fr; envelope-from="rich@annexia.org"; x-sender="postmaster@furbychan.cocan.org"; x-conformance=sidf_compatible X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgkFAJdWllFQRFuw/2dsb2JhbABbgwjCAH4WdIIfAQEEATpECwshExIPBSghAYgYCr1Lhh2HTYE+gnNhA5c3kUGDEDuBLQ X-IPAS-Result: AgkFAJdWllFQRFuw/2dsb2JhbABbgwjCAH4WdIIfAQEEATpECwshExIPBSghAYgYCr1Lhh2HTYE+gnNhA5c3kUGDEDuBLQ X-IronPort-AV: E=Sophos;i="4.87,693,1363129200"; d="scan'208";a="17960641" Received: from furbychan.cocan.org ([80.68.91.176]) by mail2-smtp-roc.national.inria.fr with ESMTP/TLS/AES256-SHA; 17 May 2013 18:16:01 +0200 Received: from rich by furbychan.cocan.org with local (Exim 4.72) (envelope-from ) id 1UdNK3-0006Hj-Uw; Fri, 17 May 2013 17:15:59 +0100 Date: Fri, 17 May 2013 17:15:59 +0100 From: "Richard W.M. Jones" To: Francois Berenger , caml-list@inria.fr Message-ID: <20130517161559.GE32353@annexia.org> References: <20130514164504.GA9228@annexia.org> <5192DFA0.2080706@riken.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5192DFA0.2080706@riken.jp> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: [Caml-list] ANNOUNCE: ocaml-bitstring 2.0.4 On Wed, May 15, 2013 at 10:06:40AM +0900, Francois Berenger wrote: [...] I'm not familiar with the problem domain you mention, however: > I mean, the bitstring can be huge, like millions of bits? They can be as long as strings in OCaml. Hence there's a limit of 16 MBytes on 32 bit architectures. It's (generally) advisable to use a 64 bit machine, and bitstrings are effectively unlimited on 64 bit. Sub-bitstrings are implemented efficiently, without copying. Bitstring matching complexity depends on the number of fields you want to match, not (at least, not obviously that I can see) on the size of fields or the size of the bitstring. If you write a bitmatch that matches a really massive number of fields [thousands?], then you'll probably run into problems with the OCaml compiler, since the bitstring macros can generate a great deal of code behind the scenes. It is a lot more efficient if the fields are aligned and the offset of each field is known at compile time. However it still works, inefficiently, even if they are not. I use bitstring a lot to parse files. However I don't usually load up the whole file into a single bitstring, but instead seek + read parts of the file. But that's just for programming convenience; it should be possible to bitmatch over subbitstrings of a whole file if you find that easier. Rich.