Ikarus Scheme

Reported memory usage mismatch between "ps" and "time"

Asked by Andreas Rottmann

I'm hacking on a GTK+ binding[0] for ikarus, and found the following discrepancy between the memory usage report of "time" from the (ikarus) library and what the UNIX "ps" command reports, when running this code:

(import (rnrs base)
         (rnrs control)
        (only (ikarus) collect nanosleep time)
        (sbank typelib))

(read) ;; At this point, "ps v" reports "72133 46816" (70.4 MB virtual, 45.7 MB resident)
(let ((main
       (time-it
        (let ()
          (typelib-import (prefix (only ("Gtk" #f) main) gtk-))
          gtk-main))))
  (main)) ;; At this point, "ps v" reports "146673 49060" (143.2 MB virtual, 47.9 MB resident)

The output from "time" is:

running stats for (let () (typelib-import (prefix (only ("Gtk" #f) main) gtk-)) gtk-main):
    no collections
    20 ms elapsed cpu time, including 0 ms collecting
    17 ms elapsed real time, including 0 ms collecting
    1402960 bytes allocated

So that's an allocation of 1.3 MB, which roughly corresponds to the increase in resident size, and seems pretty decent, given what the code does behind the scenes. The increase in virtual size, however, is a whopping 72.8 MB, or a doubling in size. This also doesn't go down when calling (collect) in a loop with sleeps inbetween. So I wonder: why does ikarus claim so much memory it *apparently* (I'm taking a wide shot here) doesn't need?

This testing was done on a AMD Athlon(tm) 64 X2 Dual Core Processor 4600+ with 4GB RAM running Ikarus Scheme version 0.0.3+ (revision 1628, build 2008-10-17, 64-bit) on top of an up-to-date Debian GNU/Linux sid (unstable) installation.

[0] Actually, it's using the gobject-introspection typelib data, so there's no specific scheme code per binding, and no C code either :-), so this should eventually open the whole GNOME stack (everything that's GObject-based) and other stuff probably too, as one should be able to generate typelibs from most reasonable C headers.

Question information

Language:
English Edit question
Status:
Solved
For:
Ikarus Scheme Edit question
Assignee:
No assignee Edit question
Solved by:
Andreas Rottmann
Solved:
Last query:
Last reply:
Revision history for this message
Abdulaziz Ghuloum (aghuloum) said : 		#1

On Oct 17, 2008, at 2:29 AM, Andreas Rottmann wrote:
> So I wonder: why does ikarus claim so much memory it *apparently*
> (I'm taking a wide shot here) doesn't need?

Ikarus does not immediately unmap the memory it does not need,
instead, there is a cache (of a fixed size: 16MB I believe) that it
maintains and never releases. This allows it to quickly unmap and
map single pages without going to the OS. But I don't think this
explains it.

The most likely cause is the segment table and card table that ikarus
uses to know the types of objects in each segment and whether it
contains backward pointers. This was initially designed for 32-bit
virtual address space. Under 64-bit, the size of that table might be
becoming too big, and it might have to change to a 2-level table.

While writing this, I have been running the GC in a loop for more
than 3 million collections now, and the size does not seem to grow.
So, I don't know if this is a problem, but make sure you do bring it
to my attention if this allocation does cause you any performance (or
other) problems.

> This testing was done on a AMD Athlon(tm) 64 X2 Dual Core
> Processor 4600+ with 4GB RAM running Ikarus Scheme version 0.0.3+
> (revision 1628, build 2008-10-17, 64-bit) on top of an up-to-date
> Debian GNU/Linux sid (unstable) installation.
>
> [0] Actually, it's using the gobject-introspection typelib data, so
> there's no specific scheme code per binding, and no C code
> either :-), so this should eventually open the whole GNOME stack
> (everything that's GObject-based) and other stuff probably too, as
> one should be able to generate typelibs from most reasonable C
> headers.

That's cool. Run-time type information, when present, makes it a
breeze to write generic wrappers. Let me know about the progress of
your project.

Aziz,,,

Revision history for this message
Andreas Rottmann (rotty) said : 		#2

> So, I don't know if this is a problem, but make sure you do bring it
> to my attention if this allocation does cause you any performance (or
> other) problems.
>
Thanks for the quick answer!

I will see if this starts to cause any problems; when I had something rudimentary
working I decided to take a quick peek into the performance, and was surprised
to see this huge ikarus process lingering around.

Also, I tried this on x86-32, and there the memory requirements were about as one would expect.

> That's cool. Run-time type information, when present, makes it a
> breeze to write generic wrappers. Let me know about the progress of
> your project.

Well, I've just kinda reached a "milestone"; one is able to open a GTK+ window, with a little bit of cheating (gtk_init has a complicated signature, so I bound that manually); the (still extremely minimal) example is at http://download.gna.org/spells/darcs/r6rs/sbank/examples/typelib.sps .