how to explain this memory usage

[Alex Martelli]

"Remco Gerlich" <scarblac at pino.selwerd.nl> wrote in message
news:slrn9egcc7.3cs.scarblac at pino.selwerd.nl...
    [snip]
> Python can free the memory, but that doesn't mean it's available for
another
> program then. On some operating systems (Windows) it simply means that
this
> program can re-use the memory. So programs don't grow smaller, only
bigger.
>
> Not a Python issue, anyway. Simply the way the OS does it.

In Windows NT, at least, it's perfectly possible for a
program to give whole pages back to the operating system
so that other programs can reuse the memory (basically
the space on the paging file -- *physical* memory is of
course reused all the time, and *address space* is quite
separate between processes anyway).  The _C runtime
library_ may or may not take advantage of this, but
MSVCRT.DLL does for large allocations.  To confirm,
make this C++ source into an EXE:

#include <stdio.h>
#include <stdlib.h>

void step()
{
    printf("Press Enter to continue: ");
    char buf[80];
    gets(buf);
}

int main(int argc, char* argv[])
{
    printf("Before allocating\n");
    step();
    void* pc = malloc(1000*1000);
    printf("After allocating a MB\n");
    step();
    free(pc);
    printf("After freeing it\n");
    step();

    return 0;
}

Now run it, examining process resources (e.g. with the
NT Task Manager) at each step.  You'll see memory
consumption as seen from the OS jumping up, then
jumping down again.  Small allocations _can_ at times
be blocked into large ones for this purpose, too --
but not always (if even a tiny bit remains allocated
inside one 8192-byte page, the whole page will not be
given back... only if ALL the bits are 'freed' can
the whole page be a candidate for "giving it back to
the OS" -- else, THAT page can only be "reused"
within that single process).


Alex