Re: Sys.argv with interpreter and compiler

Re: Sys.argv with interpreter and compiler

[Damien Doligez]

>From: Markus Mottl <mottl@miss.wu-wien.ac.at>

>I just wondered, whether it is intentional behaviour that the array
>of command line arguments ("Sys.argv") is treated exactly the same way
>under the interpreter and within executables.

Do you mean native-compiled executables versus byte-code, or compiled
programs versus the toplevel system ?

If the former, it's a bug.  If the latter, the toplevel was not really
designed for this use, and you can use Sys.interactive to tell which
case you are in.

-- Damien

Re: Sys.argv with interpreter and compiler

[Damien Doligez]

>From: Jacques GARRIGUE <garrigue@kurims.kyoto-u.ac.jp>

>Particularly, since there is no way to change the contents of
>Sys.argv, this means that you cannot easily use the Arg module in a
>caml script. Silly.

Sys.argv is an array, so it is mutable like all other arrays and you
can change its contents.  What's harder to do is to change its length,
although you could do it with Obj.truncate (not recommended in
general, except for people who know what they're doing, as usual with
Obj).

-- Damien

Re: Sys.argv with interpreter and compiler

[Jacques GARRIGUE]

From: Damien Doligez <Damien.Doligez@inria.fr>

> >From: Jacques GARRIGUE <garrigue@kurims.kyoto-u.ac.jp>
> 
> >Particularly, since there is no way to change the contents of
> >Sys.argv, this means that you cannot easily use the Arg module in a
> >caml script. Silly.
> 
> Sys.argv is an array, so it is mutable like all other arrays and you
> can change its contents.  What's harder to do is to change its length,
> although you could do it with Obj.truncate (not recommended in
> general, except for people who know what they're doing, as usual with
> Obj).

Woh, that's subtle.
I was not aware of this new Obj operation.

So this means that you can add the following code at the beginning of
your scripts, and they will then work both compiled or executed by the
toplevel.

let _ =
  let ofs = ref 1 in
  while !ofs < Array.length Sys.argv
     && String.length Sys.argv.(!ofs) > 0 && Sys.argv.(!ofs).[0] = '-'
  do if Sys.argv.(!ofs) = "-I" then ofs := !ofs + 2 else incr ofs done;
  if !Sys.interactive && !ofs < Array.length Sys.argv then begin
    Array.blit Sys.argv !ofs Sys.argv 0 (Array.length Sys.argv - !ofs);
    Obj.truncate (Obj.repr Sys.argv) (Array.length Sys.argv - !ofs);
    Sys.interactive := false
  end

Any pitfall ?
(Except that this depends heavily on the code in arg.ml/topmain.ml)
This could be a library function, Sys.script_compatible : unit -> unit.

	Jacques
---------------------------------------------------------------------------
Jacques Garrigue      Kyoto University     garrigue at kurims.kyoto-u.ac.jp
		<A HREF=http://wwwfun.kurims.kyoto-u.ac.jp/~garrigue/>JG</A>

Sys.argv with interpreter and compiler

[Markus Mottl]

Hello,

I just wondered, whether it is intentional behaviour that the array
of command line arguments ("Sys.argv") is treated exactly the same way
under the interpreter and within executables.

E.g. I want to emit an error message that includes the name of the
executable or, if the interpreter is used, the name of the script.

Wouldn't it be logically more consistent to pass the truncated array
of arguments to the script under the interpreter so that the program
always gets its name on index 0 - no matter whether it is compiled
or interpreted? - With the current version it gets the name of the
interpreter on this position.

Best regards,
Markus Mottl

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl

Re: Sys.argv with interpreter and compiler

[Xavier Leroy]

> E.g. I want to emit an error message that includes the name of the
> executable or, if the interpreter is used, the name of the script.
> 
> Wouldn't it be logically more consistent to pass the truncated array
> of arguments to the script under the interpreter so that the program
> always gets its name on index 0 - no matter whether it is compiled
> or interpreted?

Yes, it would be more consistent, but that's exactly what it does
currently.  At least, that's what a quick test under Linux shows.

> - With the current version it gets the name of the
> interpreter on this position.

That's surprising.  On which operating system do you see this
behavior?  The treatment of argv[0] in C w.r.t. #! scripts differs
between various versions of Unix, but we tried to compensate for this
in the OCaml bytecode interpreter.

- Xavier Leroy

Re: Sys.argv with interpreter and compiler

[Markus Mottl]

> > Wouldn't it be logically more consistent to pass the truncated array
> > of arguments to the script under the interpreter so that the program
> > always gets its name on index 0 - no matter whether it is compiled
> > or interpreted?
> 
> Yes, it would be more consistent, but that's exactly what it does
> currently.  At least, that's what a quick test under Linux shows.
>
> > - With the current version it gets the name of the
> > interpreter on this position.
> 
> That's surprising.  On which operating system do you see this
> behavior?  The treatment of argv[0] in C w.r.t. #! scripts differs
> between various versions of Unix, but we tried to compensate for this
> in the OCaml bytecode interpreter.

My explanation may probably be misconceived - maybe "interpreted" means
"interpreted by the byte code interpreter" to you whereas I use "compiled"
for byte code and native code and "interpreted" if I call the interactive
toplevel with a file argument.

Anyhow, I have made a test on two systems (Intel/Linux and Alpha/Digital
Unix) with ocaml-2.02-2. Both systems behaved exactly the same way, but
I got three different outputs for the three ways to execute the program.

The output of this program (bla.ml):

  print_endline Sys.argv.(0)

yielded:

interpreted:             /home/mottl/mysys/bin/ocaml
compiled to byte code:   ./bla
compiled to native code: bla

The difference between the byte code and native code version is probably
not so big a problem. But it would be useful for the development process
if the "interpreted" version would pass the arguments at the same index
in the argument vector.

As far as I remember, making OCaml (at least under Unix) a "true"
scripting-language (=with human-readable "#!"-scripts) is not so easy to
achieve: only binaries may be used as interpreters of "#!"-scripts, which
is not currently possible with the way the toplevel "ocaml" is designed -
it needs to be a byte code file. Are there already any convenient ways
around this problem?

Using byte code for scripting is not so comfortable and I think that
OCaml would give a wonderful language for "true" scripting...

Best regards,
Markus Mottl

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl

Re: Sys.argv with interpreter and compiler

[Pierre Weis]

> As far as I remember, making OCaml (at least under Unix) a "true"
> scripting-language (=with human-readable "#!"-scripts) is not so easy to
> achieve: only binaries may be used as interpreters of "#!"-scripts, which
> is not currently possible with the way the toplevel "ocaml" is designed -
> it needs to be a byte code file. Are there already any convenient ways
> around this problem?
> Using byte code for scripting is not so comfortable and I think that
> OCaml would give a wonderful language for "true" scripting...

I use a very simple way to achieve this: just tell the unix system to
execute the right ocaml interpreter, then I execute the file as
usual.

For instance:

pauillac:~$ cat > essai
#!/usr/local/bin/ocaml

print_string "Hello world!"; print_newline();;
exit 0;;
^D
pauillac:~$ chmod a+x ./essai
pauillac:~$ ./essai
Hello world!

Normally I use a special version of ocaml, with the regexp and unix
libraries linked, but it's the idea: just write plain caml code and
interpret it ! In many cases it is fast enough for what I need!

What do you think of this completely interpreted approach ?

Best regards,

Pierre Weis

INRIA, Projet Cristal, Pierre.Weis@inria.fr, http://cristal.inria.fr/~weis/

Re: Sys.argv with interpreter and compiler

[Fabrice Le Fessant]

> achieve: only binaries may be used as interpreters of "#!"-scripts, which
> is not currently possible with the way the toplevel "ocaml" is designed -
> it needs to be a byte code file. Are there already any convenient ways

Maybe you can use "ocamlmktop -custom -o ocaml" to generate a
binary version of ocaml ...

- Fabrice

Re: Sys.argv with interpreter and compiler

[Sven LUTHER]

On Fri, Jul 02, 1999 at 02:39:16AM +0200, Pierre Weis wrote:
> > As far as I remember, making OCaml (at least under Unix) a "true"
> > scripting-language (=with human-readable "#!"-scripts) is not so easy to
> > achieve: only binaries may be used as interpreters of "#!"-scripts, which
> > is not currently possible with the way the toplevel "ocaml" is designed -
> > it needs to be a byte code file. Are there already any convenient ways
> > around this problem?
> > Using byte code for scripting is not so comfortable and I think that
> > OCaml would give a wonderful language for "true" scripting...
> 
> I use a very simple way to achieve this: just tell the unix system to
> execute the right ocaml interpreter, then I execute the file as
> usual.
> 
> For instance:
> 
> pauillac:~$ cat > essai
> #!/usr/local/bin/ocaml
> 
> print_string "Hello world!"; print_newline();;
> exit 0;;
> ^D
> pauillac:~$ chmod a+x ./essai
> pauillac:~$ ./essai
> Hello world!

I don't get the same, why :


sh-2.02$ cat essay
#!/usr/local/bin/ocaml
print_string "Hello world!"; print_newline();;
exit 0;;
^D
sh-2.02$ ./essai
./essai: line 2: syntax error near unexpected token `;'
./essai: line 2: `print_string "Hello world!"; print_newline();;'

Friendly,

Sven LUTHER

Re: Sys.argv with interpreter and compiler

[Markus Mottl]

> I don't get the same, why :
> 
> sh-2.02$ cat essay
> #!/usr/local/bin/ocaml
> print_string "Hello world!"; print_newline();;
> exit 0;;
> ^D
> sh-2.02$ ./essai
> ./essai: line 2: syntax error near unexpected token `;'
> ./essai: line 2: `print_string "Hello world!"; print_newline();;'

Your toplevel "/usr/local/bin/ocaml" is obviously compiled to byte code.
Take a look at it with "less" and you will see that the first line of
this executable is actually "#!/home/mottl/mysys/bin/ocamlrun".

As far as I know it is not allowed on any unix system to run scripts
whose interpreter is a script itself. This would make it much easier to
replace this interpreter with something evil - e.g. some kind of wrapper
that executes unfriendly commands under you UID and continues with the
"true" interpreter.

Try the difference like this:

  ocamlmktop -o newtop

Now place replace "#!/usr/local/bin/ocaml" with the path to the newly
created "newtop" and run "essay" again - you will again receive an error
message, because the system will run the script with your current shell
instead with "newtop" which is not a "true" binary.

Then execute:

  ocamlmktop -custom newtop

This will generate a native code (=true binary) toplevel. You might
want to link useful native code libraries to it, eg. "unix", "str", even
"threads" and graphics work just fine!

If you run "essay" again, it will work out of the box...

I have linked my own "toplevel" like this (under Linux -> native threads
supported!):

ocamlmktop -thread -custom -o mocaml \
  graphics.cma -cclib -lgraphics -ccopt -L/usr/X11R6/lib -cclib -lX11 \
  unix.cma -cclib -lunix \
  str.cma -cclib -lstr \
  threads.cma -cclib -lthreads -cclib -lunix -cclib -lpthread

This allows me to run virtually every kind of Ocaml-script from the shell.

Try "mocaml" with this little program under X-windows:

  open Graphics
  open Random

  let main () =
    open_graph "";
    let sx, sy = size_x (), size_y () in
    let maxr = min (sx lsr 2) (sy lsr 2) and maxc = 256 in
    while not (key_pressed ()) do
      let x, y, r, color = int sx, int sy, int maxr,
                           rgb (int maxc) (int maxc) (int maxc) in
      set_color color; fill_circle x y r
    done

  let _ = Printexc.catch main ()

Nice ;-)

Best regards,
Markus Mottl

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl

Re: Sys.argv with interpreter and compiler

[Gerd Stolpmann]

On Mon, 05 Jul 1999, Markus Mottl wrote:
>> I don't get the same, why :
>> 
>> sh-2.02$ cat essay
>> #!/usr/local/bin/ocaml
>> print_string "Hello world!"; print_newline();;
>> exit 0;;
>> ^D
>> sh-2.02$ ./essai
>> ./essai: line 2: syntax error near unexpected token `;'
>> ./essai: line 2: `print_string "Hello world!"; print_newline();;'
>
>Your toplevel "/usr/local/bin/ocaml" is obviously compiled to byte code.
>Take a look at it with "less" and you will see that the first line of
>this executable is actually "#!/home/mottl/mysys/bin/ocamlrun".

There is another way to get around this:

#!/bin/sh
# (* 
exec ocaml "$0" "$@"
*) directory ".";;
print_string "Hello world!"; print_newline();;
exit(0);;

It does not matter whether "ocaml" is itself a script or not, and you 
do not need to specify the interpreter with an absolute file name;
/bin/sh always exists. The solution is a bit tricky: The first lines
are interpreted by both /bin/sh and ocaml.

	# (* 

The shell sees a comment line because of #. Ocaml sees the beginning of
a directive with a comment between # and the name of the directive
which is allowed.

	exec ocaml "$0" "$@"

This line is only seen by the shell, as ocaml thinnks it is a comment.
It causes that the process switches over to run ocaml with the given
parameters. The shell stops here and does not read further.

	*) directory ".";;

The end of the comment and the directive. #directory "." has no effect,
as "." is already in the search path.


>As far as I know it is not allowed on any unix system to run scripts
>whose interpreter is a script itself. This would make it much easier to
>replace this interpreter with something evil - e.g. some kind of wrapper
>that executes unfriendly commands under you UID and continues with the
>"true" interpreter.

That's what they tell you. I believe that kernel programmers do not like
recursions at all. (There are actually some serious problems with recursions
in kernel mode, for example recursive mutexes.)

Gerd
--
----------------------------------------------------------------------------
Gerd Stolpmann      Telefon: +49 6151 997705 (privat)
Viktoriastr. 100             
64293 Darmstadt     EMail:   Gerd.Stolpmann@darmstadt.netsurf.de (privat)
Germany                     
----------------------------------------------------------------------------

Re: Sys.argv with interpreter and compiler

[Jacques GARRIGUE]

From: Markus Mottl <mottl@miss.wu-wien.ac.at>
> My explanation may probably be misconceived - maybe "interpreted" means
> "interpreted by the byte code interpreter" to you whereas I use "compiled"
> for byte code and native code and "interpreted" if I call the interactive
> toplevel with a file argument.

I had the same problem as you, and indeed find it disturbing.
Particularly, since there is no way to change the contents of
Sys.argv, this means that you cannot easily use the Arg module in a
caml script. Silly.

> As far as I remember, making OCaml (at least under Unix) a "true"
> scripting-language (=with human-readable "#!"-scripts) is not so easy to
> achieve: only binaries may be used as interpreters of "#!"-scripts, which
> is not currently possible with the way the toplevel "ocaml" is designed -
> it needs to be a byte code file. Are there already any convenient ways
> around this problem?

That part is not really a problem.
ocaml does not need to be a code file. You can perfectly link it with
the -custom option.  This is actually what you do when you add the
Unix library, which is a minimum to write scripts...
I suppose this is Pierre did, and he never realized that there was a
potential problem here.

Actually I use this scripting approach when developping libraries:
this avoids having to compile small test examples.

For scripting, there is however another potential problem: if your
script becomes a bit long, you should really think about compiling it,
typechecking becoming slow.  The multiple VM is useful then.

Regards,

	Jacques
---------------------------------------------------------------------------
Jacques Garrigue      Kyoto University     garrigue at kurims.kyoto-u.ac.jp
		<A HREF=http://wwwfun.kurims.kyoto-u.ac.jp/~garrigue/>JG</A>

Re: Sys.argv with interpreter and compiler

[Markus Mottl]

> > As far as I remember, making OCaml (at least under Unix) a "true"
> > scripting-language (=with human-readable "#!"-scripts) is not so easy to
> > achieve: only binaries may be used as interpreters of "#!"-scripts, which
> > is not currently possible with the way the toplevel "ocaml" is designed -
> > it needs to be a byte code file. Are there already any convenient ways
> > around this problem?

Sorry for that (it was well past midnight when I wrote this). It is of
course possible to make a binary version of the toplevel with "-custom"
- I mixed this up with the debugger, which exists as byte code version
only. It didn't even come to me to try the "#!"-version (!?). Maybe I
should stop going to bed this late...

The problem with the argument vector still remains, of course: instead
of the script name, the path to the toplevel is given on index 0.

> For scripting, there is however another potential problem: if your
> script becomes a bit long, you should really think about compiling it,
> typechecking becoming slow.  The multiple VM is useful then.

Another solution would be to make parts of the file which hardly change
a separate module, compile them to binary code and link them with
the toplevel.  So only the parts that are considered more "volatile"
stay in the script.

The possibility to generate one's own "interpreter" is really a very
interesting feature of OCaml. Thus, one can nearly arbitrarily speed up
scripts by linking more and more parts with the toplevel...

Best regards,
Markus

-- 
Markus Mottl, mottl@miss.wu-wien.ac.at, http://miss.wu-wien.ac.at/~mottl