Re: [Caml-list] Efficient scanning of large strings from files

[oliver <oliver@first.in-berlin.de>]

On Fri, Mar 16, 2012 at 02:03:38PM +0100, Philippe Veber wrote:
> Dear camlers,
> 
> Say that you'd like to search a regexp on a file with lines so long that
> you'd rather not load them entirely at once. If you can bound the size of a
> match by k << length of a line, then you know that you can only keep a
> small portion of the line in memory to search the regexp. Typically you'd
> like to access substrings of size k from left to right. I guess such a
> thing should involve buffered inputs and avoid copying strings as much as
> possible. My question is as follows: has anybody written a library to
> access these substrings gracefully and with decent performance?
> Cheers,
>   Philippe.
[...]


I think, the regexp itself also has an impact on
how fast and/or how easy this can be achieved.

The more complex the Regexp, the more ressources
you will need.

If you can make your regexp becoming boult down to something
easy parseable, the length of lines might be of no importance.


Ciao,
   Oliver