mimicking a file in memory
Diez B. Roggisch
deets at nospam.web.de
Tue Nov 20 23:58:56 CET 2007
> On Nov 20, 2:06 pm, Grant Edwards <gra... at visi.com> wrote:
>> On 2007-11-20, Jarek Zgoda <jzg... at o2.usun.pl> wrote:
>>>> Here is my dilemma: I don't want to copy the files into a
>>>> local directory for mutagen's sake, only to have to remove
>>>> them afterward. Instead, I'd like to load the files into
>>>> memory and still be able to hand the built-in "file" function
>>>> a filename to access the file in memory.
>>>> Any ideas on how to do this?
>> By "memory" I presume you mean virtual memory? RAM with
>> disk-blocks as backing store? On any real OS, tempfiles are
>> just RAM with disk-blocks as backing store.
>> Sound similar? The only difference is the API used to access
>> the bytes. You want a file-I/O API, so you can either use the
>> extensively tested and and highly optimized filesystem code in
>> the OS to make disk-backed-RAM look like a file, or you can try
>> to write Python code that does the same thing.
>> Which do you think is going to work faster/better?
>> [The kernel is generally better at knowing what needs to be in
>> RAM than you are -- let it do its job.]
>> IOW: just use a temp file. Life will be simple. The bytes
>> probably won't ever hit the platters (if they do, then that
>> means they would have the other way too).
>> Grant Edwards grante Yow! It's a hole all the
>> at way to downtown Burbank!
> Thanks all.
> Grant, are temp files automatically put into ram for all linux
> distros? at any rate, i could set up ram disk. much better solution
> than using python...except that i've never done a ram disk before.
> more reading to do...
You misunderstood Grant. Let the OS decide what needs to be dumped to
the HD or not - instead of creating a RAM-disk eating up precious ram.
All modern OS, including Linux, have hard-disc-caches in RAM. Which your
downloaded file ends in, whilst being stored to the HD in the background
- without affecting your performance. If you then pass it to some other
process that reads from the file, it will be fed from the HD-cache - fast.
So - if it makes your life easier to use tempfiles because then you have
an actual filename, use them and don't mind.
More information about the Python-list