File Read Cache - How to purge?
mugi at pinesalad.net
Tue Aug 21 11:29:14 CEST 2007
> What do you mean by so little memory. It (the whole file) is cached by the
> operating system totally independent of your program.
Please note I already stated it was more than likely by the OS and
noted the tests to confirm that.
> It (the whole file) is cached by the operating system totally independent
> of your program, so the memory used does of course not show up in the memory
> stats of your program... <snip>
In this case the OS is Windows and monitoring the memory usage in Task
Manager, not through the script. The entire 759MB file is not being
cached in memory and only 2MB of memory is used when the script runs.
You can see in the example script that I'm not storing the file in
memory (buf is "overwritten" at each read(size)) and no memory stats
are being kept there. Not sure where I might have eluded otherwise,
but hope this clears that up.
> > 2. Is there anyway to somehow to take advantage of this "caching" by
> > initializing it without reading through the entire file first?
> You mean reading the file without actually reading it!? :-)
Think you misunderstood.
What the "tests" are eluding to is:
a. The whole file itself is NOT being cached in memory.
b. If there is mechanism to which it is "caching" something (which
obviously isn't the whole file itself), why not possibly take
advantage of it?
And sometimes there can be "tricks" to "initializing" before actually
read/writing a file to help improve some performance (and not
necessarily via a cache).
More information about the Python-list