[Numpy-discussion] multiprocessing shared arrays and numpy
gael.varoquaux at normalesup.org
Sun Mar 7 14:03:21 EST 2010
On Sun, Mar 07, 2010 at 07:00:03PM +0000, René Dudfield wrote:
> 1. Mmap'd files are useful since you can reuse disk cache as program
> memory. So large files don't waste ram on the disk cache.
I second that. mmaping has worked very well for me for large datasets,
especialy in the context of reducing memory pressure.
More information about the NumPy-Discussion