[Numpy-discussion] About the npz format

[onefire]

Interesting! Using sync() as you suggested makes every write slower,  and
it decreases the time difference between save and savez,
so maybe I was observing the 10 times difference because the file system
buffers were being flushed immediately after a call to savez, but not right
after a call to np.save.

I think your workaround might help, but a better solution would be to not
use Python's zipfile module at all. This would make it possible to, say,
let the user choose the checksum algorithm or to turn that off.
Or maybe the compression stuff makes this route too complicated to be worth
the trouble? (after all, the zip format is not that hard to understand)

Gilberto



On Thu, Apr 17, 2014 at 6:45 PM, Valentin Haenel <valentin at haenel.co> wrote:

> Hello,
>
> * Valentin Haenel <valentin at haenel.co> [2014-04-17]:
> > As part of bloscpack.sysutil I have wrapped this to be available from
> > Python (needs root though). So, to re-rurn the benchmarks, doing each
> > one twice:
>
> Actually, I just realized, that doing a ``sync`` doesn't require root.
>
> my bad,
>
> V-
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140417/e7cdf1d2/attachment.html>