[Tutor] Load Entire File into memory

[Oscar Benjamin]

On 5 November 2013 13:20, Amal Thomas <amalthomas111 at gmail.com> wrote:
> On Mon, Nov 4, 2013 at 10:00 PM, Steven D'Aprano <steve at pearwood.info>
> wrote:
>>
>
>>
>> import os
>> filename = "YOUR FILE NAME HERE"
>> print("File size:", os.stat(filename).st_size)
>> f = open(filename)
>> content = f.read()
>> print("Length of content actually read:", len(content))
>> print("Current file position:", f.tell())
>> f.close()
>>
>>
>> and send us the output.
>
>
>  This is the output:
>    File size: 50297501884
>    Length of content actually read: 50297501884
>    Current file position: 50297501884
> This Code used 61.4 GB RAM and 59.6 GB swap (I had ensured that no other
> important process were running in my server before running this :D)

If you are using the swap then this will almost certainly be the
biggest slowdown in your program (and any other program on the same
machine). If there is some way to reorganise your code so that you
don't need to load everything into memory then (if you do it right)
you should be able to make it *much* faster.


Oscar