[AstroPy] PyFITS memory usage
glenn.caltech at gmail.com
Fri May 13 15:49:25 EDT 2011
I am tracking memory usage using top which shows the "RES" and "SHR" columns
growing steadily. Interestingly, using ksysguard, I see the blue
"Application memory" curve staying roughly constant as I would expect.
However, when the memory usage reported by top gets near the amount of
physical memory on the system, I do see the system start to swap-thrash. So
I believe the usage reported by top is accurate.
On Fri, May 13, 2011 at 10:20 AM, Brandon Craig Rhodes <
brandon.craig.rhodes at gmail.com> wrote:
> G Jones <glenn.caltech at gmail.com> writes:
> > I am processing PSRFITS files which are ~40 GB and have a table with
> > the data in it. Since the files are so large, I open them with
> > memmap=True. I would expect the following simple script to have a
> > constant memory footprint, but instead I see continuously growing
> > memory usage. ... I can see that the data is being read in from disk
> > as it should be, but I don't expect the memory usage to increase,
> > since x is constantly overwritten.
> What are you using to measure its memory use - what tool is showing you
> that its memory use is going up? It will help us answer if we know
> which memory-use metric (of the several offered by tools like "ps" and
> "top") are the ones you see increasing.
> Brandon Craig Rhodes brandon at rhodesmill.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the AstroPy