Interactive technical computing

[Jon Harrop <jon@ffconsultancy.com>]

Being a user of both OCaml and Mathematica, playing with the new F# language 
from Microsoft and watching tutorial videos about VPython:

  http://showmedo.com/videos/series?name=pythonThompsonVPythonSeries

has given me a lot of inspiration about interactive technical computing 
environments. This class of applications is hugely useful for working 
scientists and engineers because it lets you slice and dice your data in 
interesting ways whilst also giving you visual throwback and even letting you 
do some fancy visualisations.

For example, I'm in the process of updating my ray tracer language comparison:

  http://www.ffconsultancy.com/languages/ray_tracer/index.html

and I'm using a mix of OCaml (to fire off compilation and execution commands) 
and Mathematica (to dissect the results, compute verbosity using regexps and 
plot graphs):

  http://www.ffconsultancy.com/tmp/mathematica.png

Mathematica's equivalent of the OCaml top-level is called a notebook. It 
provides expression input and result output, just like OCaml, but integrates 
graphics, adds typesetting and lots of mathematical functions. However, it is 
widely used for more general purpose programming despite being very slow.

Using F# from Visual Studio 2005 provides some of this functionality. The 
following screenshots illustrate 2D and 3D graphics spawned from an F# 
interactive session using a little of my own code and DirectX/ComponentsXtra:

  http://www.ffconsultancy.com/tmp/fs_xygraph.png
  http://www.ffconsultancy.com/tmp/fs_3dplot.png

For all non-trivials examples in F# it is necessary to spawn a separate thread 
to handle the GUI of the visualization, or the GUI will hang when the 
top-level is doing an intensive computation.

I think F# has a great future because of its ability to spawn visualizations 
from a running interactive session. Expensive commercial offerings like 
Matlab and Mathematica are ok when you're doing something they have built-in 
(e.g. a Fourier transform) but when you're problem is not trivially 
decomposed into their built-in operators (e.g. a wavelet transform), F# and 
OCaml are typically 2-5x faster, and when you must resort to more general 
purpose programming F# and OCaml are often 100x faster.

However, there is a lot of work to be done in getting competitive charting and 
visualization tools into F# and I'm thinking that OCaml could benefit from a 
joint venture here. Low-level routines would target DirectX in F# and OpenGL 
in OCaml but high-level routines could be language and platform agnostic, 
handling a scene graph that is essentially a typed version of Mathematica's 
to provide much faster graphics and even interactive visualisation 
(Mathematica is software rendered and not interactive!).

This raises several questions:

. What OCaml programs currently allow OpenGL-based visualizations to be 
spawned from the top-level?

. Has anyone tried to write an IDE that mixes OCaml code with graphics?

. Would anyone here be interested in a low-cost cross-platform technical 
computing environment based upon the OCaml and F# languages?

Obviously I'm interested in this from a commercial perspective. That looks 
easy for F# but not so easy for OCaml. Compiled OCaml+OpenGL code is not as 
portable (between machines) as F#+DirectX. Also, I can sell F# DLLs and even 
make the library available to other .NET languages (albeit with a 
significantly less productive API).

Finally, I'd like to note that operator overloading is probably the single 
biggest difference between my F# and OCaml code. The ability to apply + and - 
to many types, particularly vectors and matrices, makes this kind of work so 
much easier. Even if you have to add the odd type annotation. So I'd love to 
see a compatible implementation of overloading introduced into OCaml.

I'd like to hear everyone's opinions on this as, it seems to me, we're sitting 
on the foundations of a great technical computing system.

-- 
Dr Jon D Harrop, Flying Frog Consultancy Ltd.
OCaml for Scientists
http://www.ffconsultancy.com/products/ocaml_for_scientists