[Numpy-discussion] About the npz format

[Valentin Haenel]

Hi Gilberto,

* onefire <onefire.myself at gmail.com> [2014-04-18]:
> Interesting! Using sync() as you suggested makes every write slower,  and
> it decreases the time difference between save and savez,
> so maybe I was observing the 10 times difference because the file system
> buffers were being flushed immediately after a call to savez, but not right
> after a call to np.save.

I am happy that you found my suggestion useful! Given that the current
savez implementation first writes temporary arrays to disk and then
copies them from their temporary location to the zipfile, one might
argue that this is what causes the buffers to be flushed, since it does
more IO than the save implementation. Then again I don't really now the
gory details of the how the filesystem buffers behave and how they can
be configured.

best,

V-