[AstroPy] PyFITS memory usage

G Jones glenn.caltech at gmail.com
Fri May 13 15:49:25 EDT 2011


Hi Brandon,
I am tracking memory usage using top which shows the "RES" and "SHR" columns
growing steadily. Interestingly, using ksysguard, I see the blue
"Application memory" curve staying roughly constant as I would expect.
However, when the memory usage reported by top gets near the amount of
physical memory on the system, I do see the system start to swap-thrash. So
I believe the usage reported by top is accurate.
Glenn

On Fri, May 13, 2011 at 10:20 AM, Brandon Craig Rhodes <
brandon.craig.rhodes at gmail.com> wrote:

> G Jones <glenn.caltech at gmail.com> writes:
>
> > I am processing PSRFITS files which are ~40 GB and have a table with
> > the data in it. Since the files are so large, I open them with
> > memmap=True. I would expect the following simple script to have a
> > constant memory footprint, but instead I see continuously growing
> > memory usage. ...  I can see that the data is being read in from disk
> > as it should be, but I don't expect the memory usage to increase,
> > since x is constantly overwritten.
>
> What are you using to measure its memory use - what tool is showing you
> that its memory use is going up?  It will help us answer if we know
> which memory-use metric (of the several offered by tools like "ps" and
> "top") are the ones you see increasing.
>
> --
> Brandon Craig Rhodes   brandon at rhodesmill.org
> http://rhodesmill.org/brandon
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110513/92770783/attachment.html>


More information about the AstroPy mailing list