Re: [Caml-list] Great Programming Language Shootout Revived

[Brian Hurt <bhurt@spnz.org>]

On Fri, 18 Jun 2004, Ville-Pertti Keinonen wrote:

> Anyhow, the shootout seems, like most benchmarks, to be misleading and 
> arbitrary.  The language features compared are not really equivalent.  
> E.g. C, C++ and Ada should be approximately the same in performance for 
> code that doesn't compare their libraries or exception models.  Despite 
> doing things like disabling array index checks, it seems some of the 
> Ada benchmarks don't even use types equivalent to the C versions.

I agree.  Even worse, the majority exercise only small amounts of the 
language.  The tests array accesses, exception handling, method calls, 
nested loops, object instantiations, and string concatenation all 
explicitly test specific operations- with no regard for how often those 
options are actually implemented.  Take, for example, exception handling.  
The C version of this code uses setjmp()/longjmp().  Two functions which 
are rarely actually used in C code- the standard language pattern is to 
return an error code instead.  And other tests- notably ackerman's 
function and fibonacci numbers, are mainly dependent upon a similiarly 
small set of features (mainly speed of function calls).

"Artificial" doesn't begin to cover it.

As a side note, there is a reason why C++ is slower than C, even if you're 
compiling the exact same code.  C++ functions need to be able to handle 
exceptions and unroll the stack, even if the function itself doesn't throw 
or catch exceptions.  Even if unrolling the stack is a no-op, the function 
generally needs to be able to both be handed off from another function's 
stack unrolling, and hand off to another function's stack unrolling.  To 
remove this cost, the compiler needs to do a full program analysis, and 
basically detect "wait- you could compile this code with a C compiler, so 
I will compile this code with a C compiler!"  While this may be doable in 
theory, it's not going to happen in the real world.

Which raises another point.  Many languages, and C++ and Ocaml both
qualify, have "preferred" styles of programming, even if they admit other
styles.  For example, Ocaml "prefers" functional programming, it's the
default style, even as it allows imperitive and OO programming constructs.  
C++ "prefers" a template/OO style of coding, even as it admits C as
(basically) a proper subset.

If I were to sit down and start writting an application in Ocaml, 99% or 
more of it would be in a functional style.  How fast Ocaml is in 
imperitive or OO would not likely be all that important to the overall 
performance of my application.  Likewise, were I to write an application 
in C++, it's use templates, classes, virtual methods, etc.  If I wanted to 
write the application in C, I'd write it in C, not C++ and not Ocaml.  
Even if they involve a performance hit, they have other advantages (easier 
to write/debug code in, easier to use more advanced data structures, 
etc.).  This then raises the question of how much performance am I 
sacrificing for this increased power at the high level?

Plus there's the whole efficiency in the small vr.s efficiency in the 
large issue I mentioned earlier.  The GPLS measures almost entirely 
efficiency in the small.  Which isn't necessarily bad, just that it 
doesn't necessarily map to larger projects.

-- 
"Usenet is like a herd of performing elephants with diarrhea -- massive,
difficult to redirect, awe-inspiring, entertaining, and a source of
mind-boggling amounts of excrement when you least expect it."
                                - Gene Spafford 
Brian

-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners