[Caml-list] Arbitrarily throwing End_of_file

[Caml-list] Arbitrarily throwing End_of_file

[Michael Hoisie]

I have a file which is approximately 278,440 lines of text (more specifically, it is the result of doing 'ls -lAR /')

I was trying to write this relatively simple program to analyze it but it seems that End_of_file was thrown very early. 

To test, it, I made a simple function:  

let rec count_lines file n =
    try let str = input_line file in
    count_lines file (n + 1)
    with End_of_file -> Printf.printf "The file is %d\n lines long" n

and ran it with something like:
    let file = open_in "longfile.dat" in
    count_lines file 0

The result was 26187.

count_lines is a stripped-down version of the original function i used to analyze the file. If I used the original, it threw an End_of_file at line 23806. 

I'm running 3.07+2 on FreeBSD 4.9
any suggestions?
-mike

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Arbitrarily throwing End_of_file

[Brian Hurt]

On Sun, 9 Nov 2003, Michael Hoisie wrote:

> I have a file which is approximately 278,440 lines of text (more
> specifically, it is the result of doing 'ls -lAR /')

-l lists the file size in *bytes*, not lines.  Use 'wc -l longfile.dat' to
determine the number of lines.  If each line is ~10.6 bytes long
(including the EOLN) then a 278,000 byte file will be about 26,000 lines
long.  The -A means "almost all" (everything except . and ..), and the R
means recursive (list subdirectories as well).

> 
> I was trying to write this relatively simple program to analyze it but
> it seems that End_of_file was thrown very early.
> 
> To test, it, I made a simple function:  
> 
> let rec count_lines file n =
>     try let str = input_line file in
>     count_lines file (n + 1)
>     with End_of_file -> Printf.printf "The file is %d\n lines long" n

This function isn't tail recursive- the function's call to itself is
within a try/with block, which breaks the tail recursion.  That isn't the
problem you're hitting, but you're not far from hitting it.  I generally
hit it about 30,000 functions deep or so.  Try the following instead:

let rec count_lines file n =
    let line, eof = try (input_line file), false
                   with End_of_file -> "", true
    in
    if not eof then
        begin
            (* do something with line here *)
            count_lines file (n + 1)
        end
    else
        n

let file = open_in "longfile.dat" in
Printf.printf "The file is %d lines long.\n" (count_lines file 0)

Note that the tail recursion is now outside of the try/with block, and 
this function will work with any length file.

Brian


-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners