"High water" Memory fragmentation still a thing?
rosuav at gmail.com
Sat Oct 4 04:22:38 CEST 2014
On Sat, Oct 4, 2014 at 11:02 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> Chris Angelico wrote:
>> In theory, a high
>> level language like Python would be allowed to move objects around to
>> compact memory, but CPython doesn't do this, and there's no proof that
>> it'd really help anything anyway.
> I welcome correction, but I understand that both the JVM and the CLR memory
> managers move memory around. That's why Jython and IronPython use
> sequential integers as object IDs, since memory locations are not fixed.
Right; I should have made it clearer that there's no proof that it'd
help anything *in CPython*. Removing the GIL is periodically proposed,
too, but there's no proof that its removal would benefit CPython; it's
not just that nobody's gotten around to writing a memory compactor for
> Ironically, the cost of restarting the process periodically is likely to be
> orders of magnitude more expensive than that of moving a few blocks of
> memory around from time to time. Especially on Windows, where starting
> processes is expensive, but even on Linux you have to shut the running
> application down, then start it up again and rebuild all the internal data
> structures that you just tore down...
Maybe. But you deal with a number of things all at once:
1) Code updates (including interpreter updates)
2) Compaction of Python objects
3) Disposal of anything that got "high level leaked" - unintended
longevity caused by a global reference of some sort
4) Cleanup of low-level allocations that don't go through the Python
etc etc etc.
So, yes, it's expensive. And sometimes it's not even possible (there's
no way to retain socket connections across a restart, AFAIK). But it's
there if you want it.
Personally, I like to keep processes running, but that's me. :)
More information about the Python-list