Implementing file reading in C/Python
siona at chiark.greenend.org.uk
Mon Jan 12 12:58:12 CET 2009
Grant Edwards <invalid at invalid> wrote:
>On 2009-01-09, Sion Arrowsmith <siona at chiark.greenend.org.uk> wrote:
>> Grant Edwards <invalid at invalid> wrote:
>>>If I were you, I'd try mmap()ing the file instead of reading it
>>>into string objects one chunk at a time.
>> You've snipped the bit further on in that sentence where the
>> OP says that the file of interest is 2GB. Do you still want to
>> try mmap'ing it?
>Sure. The larger the file, the more you gain from mmap'ing it.
>2GB should easily fit within the process's virtual memory
Assuming you're in a 64bit world. Me, I've only got 2GB of address
space available to play in -- mmap'ing all of it out of the question.
But I supposed that mmap'ing it chunk at a time instead of reading
chunk at a time might be worth considering.
\S -- siona at chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
"Frankly I have no feelings towards penguins one way or the other"
-- Arthur C. Clarke
her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump
More information about the Python-list