dynamic allocation file buffer
faltet at gmail.com
Mon Sep 15 11:34:22 CEST 2008
On 12 Set, 14:39, "Aaron \"Castironpi\" Brady" <castiro... at gmail.com>
> > A consideration of other storage formats such as HDF5 might
> > be appropriate:
> > There are, of course, HDF5 tools available for Python.
> PyTablescame up within the past few weeks on the list.
> "When the file is created, the metadata in the object tree is updated
> in memory while the actual data is saved to disk. When you close the
> file the object tree is no longer available. However, when you reopen
> this file the object tree will be reconstructed in memory from the
> metadata on disk...."
> This is different from what I had in mind, but the extremity depends
> on how slow the 'reconstructed in memory' step is. (Fromhttp://www.pytables.org/docs/manual/ch01.html#id2506782). The
> counterexample would be needing random access into multiple data
> files, which don't all fit in memory at once, but the maturity of the
> package might outweigh that. Reconstruction will form a bottleneck
Hmm, this was a part of a documentation that needed to be updated.
Now, the object tree is reconstructed in a lazy way (i.e. on-demand),
in order to avoid the bottleneck that you mentioned. I have corrected
the docs in:
Thanks for (indirectly ;-) bringing this to my attention,
More information about the Python-list