Re: [Caml-list] Measuring GC latencies for OCaml program

[pratikfegade@gmail.com]

[-- Attachment #1: Type: text/plain, Size: 7414 bytes --]

Hi,

I am not sure  if this is the right place to ask this but this being the only reference to the instrumented runtime I found, I am going to go ahead and ask.

I am trying to use the instrumented runtime as you described but it does not seem to work. I cannot compile the program with the instrumented runtime. I have compiled the compiler with the --with-instrumented-runtime flag (I have ocamlruni) in my path. I get the following error while trying to compile the way you suggested:

Configuration "true: quiet, runtime_variant(i)", line 1, characters 13-31:
Warning: tag "runtime_variant" does not expect a parameter, but is used with parameter "i"
Configuration "true: quiet, runtime_variant(i)", line 1, characters 13-31:
Warning: the tag "runtime_variant(i)" is not used in any flag declaration, so it will have no effect; it may be a typo. Otherwise use `mark_tag_used` in your myocamlbuild.ml to disable this warning.
Finished, 4 targets (0 cached) in 00:00:00.

I also tried to compile directly to native code using the command

ocamlopt -runtime-variant i main.ml

which does not give an error which I assume to mean that the instrumented runtime exists on my system. I cannot however find any log file nor does running the executable give any information on stdout or stderr.

Am I missing something here?

Thanks,
Pratik Fegade.

On Monday, May 30, 2016 at 3:49:21 PM UTC-4, Gabriel Scherer wrote:
> Dear caml-list,
> 
> You may be interested in the following blog post, in which I give
> instructions to measure the worst-case latencies incurred by the GC:
> 
>   Measuring GC latencies in Haskell, OCaml, Racket
>   http://prl.ccs.neu.edu/blog/2016/05/24/measuring-gc-latencies-in-haskell-ocaml-racket/
> 
> In summary, the commands to measure latencies look something like:
> 
>     % build the program with the instrumented runtime
>     ocamlbuild -tag "runtime_variant(i)" myprog.native
> 
>     % run with instrumentation enabled
>     OCAML_INSTR_FILE="ocaml.log" ./main.native
> 
>     % visualize results from the raw log
>     $(OCAML_SOURCES)/tools/ocaml-instr-graph ocaml.log
>     $(OCAML_SOURCES)/tools/ocaml-instr-report ocaml.log
> 
> While the OCaml GC has had a good incremental mode with low latencies
> for most workloads for a long time, the ability to instrument it to
> actually measure latencies is still in its infancy: it is
> a side-result of Damien Doligez's stint at Jane Street last year, and
> 4.03.0 is the first release in which this work is available.
> 
> A practical consequence of this youth is that the "user experience" of
> actually performing these measurements is currently very bad. The GC
> measurements are activated in an instrumented runtime variant (OCaml
> supports having several variants of the runtime available, and
> deciding which one to use for a specific program at link-time), which
> is the right design choice, but right now this variant is not built by
> default by the compiler distribution -- building it is
> a configure-time option disabled by default. This means that, as
> a user interested in doing the measurements, you have to compile an
> alternative OCaml compiler.
> Furthermore, processing the raw instrumented log requires tool that
> are also in their infancy, and are currently included in the compiler
> distribution sources but not installed -- so you have to have a source
> checkout available to use them. In contrast, GHC's instrumentation is
> enabled by just passing the option "+RTS -s" to the Haskell program of
> interest; this is superbly convenient and something we should aim at.
> 
> I discussed with Damien whether we should enable building the
> instrumented runtime by default (for example pass
> the --with-instrumented-runtime option to the opam switches people are
> using, and encourage distributions to use it in their packages as
> well). Of course there is a cost/benefit trade-off: currently
> virtually nobody is using this instrumentation, but enabling it by
> default would increase the compilation time of the compiler
> distribution for everyone. (On my machine it only adds 5 seconds to
> total build time.)
> 
> I personally think that we should aim for a rock-solid experience for
> profiling and instrumenting OCaml program enabled by default¹. It is
> worth making it slightly longer for anyone to install the compiler if
> we can make it vastly easier to measure GC pauses in our program when
> the need arises (even if it's not very often). But that is
> a discussion that should be had before making any choice.
> 
> Regarding the log analysis tools, before finding about Damien's
> included-but-not installed tools (a shell and an awk script, in the
> finest Unix tradition) I built a quick&dirty OCaml script to do some
> measurements, which can be found in the benchmark repository below. It
> would not be much more work to grow this in a reusable library to
> extract the current log format into a structured data structure -- the
> format is undocumented but the provided scripts in tools/ have enough
> information to infer the structure. Such a script/library would, of
> course, remain tightly coupled to the OCaml version, but I think it
> could be useful to have it packaged for users to play with.
> 
>   https://gitlab.com/gasche/gc-latency-experiment/blob/master/parse_ocaml_log.ml
> 
> ¹: We cannot expect users to effectively write performant code if they
> don't have the tool support for it. The fact that lazyness in Haskell
> makes it harder for users to reason about efficiency or memory usage
> has made the avaibility of excellent performance tooling *necessary*,
> where it is merely nice-to-have in OCaml. Rather ironically, Haskell
> tooling is now much better than OCaml's in this area, to the point
> that it can be easier to write efficient code in Haskell.
> 
> Three side-notes on profiling tools:
> 
> 1. `perf record --call-graph=dwarf` works fine for ocamlopt binaries
>   (no need for a frame-pointers switch), and this is documented:
>     https://ocaml.org/learn/tutorials/performance_and_profiling.html#UsingperfonLinux
> 
> 2. Thomas Leonard has done excellent work on domain-specific profiling
>    tools for Mirage, and this is the kind of tool support that I think
>    should be available to anyone out of the box.
>      http://roscidus.com/blog/blog/2014/08/15/optimising-the-unikernel/
>      http://roscidus.com/blog/blog/2014/10/27/visualising-an-asynchronous-monad/
> 
> 3. There is currently more debate than anyone could wish for around
>    a pull request of Mark Shinwell for runtime support for dynamic call
>    graph construction and its use for memory profiling.
>      https://github.com/ocaml/ocaml/pull/585
> 
> 4. Providing a good user experience for performance or space profiling
>    is a fundamentally harder problem than for GC pauses. It may
>    require specially-compiled versions of the libraries used by your
>    program, and thus a general policy/agreement across the
>    ecosystem. Swapping a different runtime at link-time is very easy.
> 
> -- 
> Caml-list mailing list.  Subscription management and archives:
> https://sympa.inria.fr/sympa/arc/caml-list
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs