Memory utilization (linux v. openbsd)
torriem at chem.byu.edu
Sun Jul 29 03:32:06 CEST 2007
nazgul at jsbsystems.com wrote:
> My problem is that the two process under OpenBSD are going to fail with
> a MemoryError becaause the size just keeps getting larger and larger.
> ulimit -d is 1G for each process.
The problem is that you can't get accurate memory use readings from top.
The reality is that your python program is probably using the same
amount of memory on both platforms (in and of itself), but due to the
way the memory management OS system works, the usage is being reported
differently on BSD. Basically you're seeing the memory footprint of the
python executable, it's data (your program) and also all of the shared
libraries that python is linked to. Of course the shared libraries are
shared, so it's not like python's using all this memory just itself.
> Any idea why the memory utilization is so much higher under OpenBSD?
> What I also find interesting is that the OpenBSD box seems to grind to a
> halt (that's probably due to swapping since there's 500M of swap space
> in use). Those processes should both be over 90% since they are CPU
> bound, and they are that way at the start of the run. Gradually, the
> system % creeps up as does idle, but there's no reason for them to be
> idle. It's a CPU bound process. (While I'm typing this top is showing
> cpu rats of 63% and 41% for the two jobs).
I can't answer your questions or solve your problem. But I just wanted
to point out that the problem is likely somewhere besides python itself.
Maybe there are settings in the OS. Who knows. My guess is the
difference in overall memory footprint is due to differences in the C
library and its dependencies. You probably would want to bring this
issue up on the OpenBSD lists as it's clear that your BSD machine is
having problems generally (trashing to death). This doesn't appear to
be a python problem.
More information about the Python-list