Re: [Caml-list] Overflow with Spacetime

[Leo White <lwhite@janestreet.com>]

[-- Attachment #1: Type: text/plain, Size: 2164 bytes --]

> Possibly I ran into a space leak while trying to find a space leak.

7GB for a 900MB profile sounds about right unfortunately. The format of
spacetime profiles makes it difficult for the viewer to do much better than
that. I hope to get around to changing the format to fix this, but probably
won't until some time next year.

I would recommend trying to get a smaller profile. Taking snapshots less
often will help, as will running the program for a shorter time (even with
the same number of snapshots). Failing that try getting access to a machine
with more memory.

On 9 August 2017 at 08:37, Kenneth Adam Miller <kennethadammiller@gmail.com>
wrote:

> Hello all,
>
>
> I wrote a particular application, and after profiling I can see that the
> garbage collector is using 95% of the time in the application and that it
> uses up to 6 gigs of memory on a 1 megabyte input file for a run that
> totals out to be about 2 minutes. A 27 kilobyte input file takes seconds to
> run, so I know that there is some explosive leak or otherwise terrible
> issue with the space complexity here. I doubled down, and got compilation
> to run to completion using _tags to make sure that there were symbols and
> profiling turned on (thanks Ivan, Drup).
>
> Additionally, I compiled my application with spacetime and ran it to
> obtain a `spacetime-<pid>` binary, which I then attempted to get a report
> on with prof_spacetime. prof_spacetime went up to about 7.3 gigs in memory
> usage before it repeatedly would die with `Killed` or `Stack overflow` as
> the error. I'm trying to figure out what else there is I can do to make
> sure that the process runs to completion. I double checked to make sure
> that I was running the prof_spacetime command built with just a 4.04.0
> compiler to ensure that it wouldn't profile itself while trying to "Process
> series". My spacetime dump is about 900 Mb, so I understand that it would
> use some memory processing it. Possibly I ran into a space leak while
> trying to find a space leak.
>
> In the meantime, I will try and find some intermediate size input to get
> something I can work on - thought I should report this.
>
>
>

[-- Attachment #2: Type: text/html, Size: 2788 bytes --]