Implementing file reading in C/Python

Sion Arrowsmith siona at
Mon Jan 12 12:58:12 CET 2009

Grant Edwards  <invalid at invalid> wrote:
>On 2009-01-09, Sion Arrowsmith <siona at> wrote:
>> Grant Edwards  <invalid at invalid> wrote:
>>>If I were you, I'd try mmap()ing the file instead of reading it
>>>into string objects one chunk at a time.
>> You've snipped the bit further on in that sentence where the
>> OP says that the file of interest is 2GB. Do you still want to
>> try mmap'ing it?
>Sure.  The larger the file, the more you gain from mmap'ing it.
>2GB should easily fit within the process's virtual memory

Assuming you're in a 64bit world. Me, I've only got 2GB of address
space available to play in -- mmap'ing all of it out of the question.

But I supposed that mmap'ing it chunk at a time instead of reading
chunk at a time might be worth considering.

\S -- siona at --
   "Frankly I have no feelings towards penguins one way or the other"
        -- Arthur C. Clarke
   her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump

More information about the Python-list mailing list