[Numpy-discussion] About the npz format
onefire.myself at gmail.com
Thu Apr 17 20:12:43 EDT 2014
I found this github issue (https://github.com/numpy/numpy/pull/3465) where
someone mentions the idea of forking the zip library.
On Thu, Apr 17, 2014 at 8:09 PM, onefire <onefire.myself at gmail.com> wrote:
> Interesting! Using sync() as you suggested makes every write slower, and
> it decreases the time difference between save and savez,
> so maybe I was observing the 10 times difference because the file system
> buffers were being flushed immediately after a call to savez, but not right
> after a call to np.save.
> I think your workaround might help, but a better solution would be to not
> use Python's zipfile module at all. This would make it possible to, say,
> let the user choose the checksum algorithm or to turn that off.
> Or maybe the compression stuff makes this route too complicated to be
> worth the trouble? (after all, the zip format is not that hard to
> On Thu, Apr 17, 2014 at 6:45 PM, Valentin Haenel <valentin at haenel.co>wrote:
>> * Valentin Haenel <valentin at haenel.co> [2014-04-17]:
>> > As part of bloscpack.sysutil I have wrapped this to be available from
>> > Python (needs root though). So, to re-rurn the benchmarks, doing each
>> > one twice:
>> Actually, I just realized, that doing a ``sync`` doesn't require root.
>> my bad,
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion