[Tutor] Load Entire File into memory

[Danny Yoo]

>
> You _must_ avoid swap at all costs here.  You may not understand the
> point, so a little more explanation: touching swap is several orders of
> magnitude more expensive than anything else you are doing in your program.
>
>     CPU operations are on the order of nanoseconds. (10^-9)
>
>     Disk operations are on the order of milliseconds.  (10^-3)
>
> References:
>
>     http://en.wikipedia.org/wiki/Instructions_per_second
>
> http://en.wikipedia.org/wiki/Hard_disk_drive_performance_characteristics
>
>

To visualize the sheer scale of the problem, see:

    http://i.imgur.com/X1Hi1.gif

which would normally be funny, except that it's not quite a joke.  :P


So you want to minimize hard disk usage as much as possible.  "Thrashing"
is precisely the situation you do not want to have when running a large
analysis.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20131104/b357acfc/attachment.html>