Efficient parallel programming without a concurrent GC

Efficient parallel programming without a concurrent GC

[Jon Harrop]

I believe I have envisaged a useful and feasible way to combine the benefits 
of OCaml's GC with efficient parallel programming without requiring a 
concurrent GC.

I have been considering the ramifications of being able to quote HLVM code in 
camlp4 macros as a DSL from within OCaml programs. That would combine OCaml's 
better toolchain with HLVM's better numerical performance and (in the future) 
efficient parallelism.

For example, a program might initialize an OCaml bigarray representing the 
physical state of a simulation or computer game environment and call into 
HLVM to have the state updated in-place using efficient parallel algorithms 
implemented as multithreaded code. The result could even be passed directly 
to OpenGL for visualization. In particular, the OCaml code responsible for 
the UI would benefit from short pause times whereas the HLVM code responsible 
for number crunching would benefit from high throughput.

This approach appears to solve almost all of the outstanding problems because 
the programmer can choose between two different ways to execute code that 
have different characteristics and the two modes of evaluation can run 
concurrently using shared state for O(1) communication. Moreover, neither 
side is burdened with the inefficiencies of a concurrent GC.

The disadvantage is that the two GCs must remain oblivious to each other so 
data passed between them must be managed manually. However, that seems like a 
small price to pay because the data will inevitably be arrays of value types 
that are easily managed.

Does that sound valuable?

-- 
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e