[SciPy-user] scipy, numeric, numarray, pytables/hdf5 & ?numeric3/scipy.base

[Francesc Altet]

A Dimarts 26 Abril 2005 15:19, George Nurser va escriure:
> I'm interested in using scipy and probably matplotlib to analyse and
> plot data from a large-scale ocean model whose data are held in hdf5
> format.
>
> The most obvious way to read (and write) data to hdf5 would seem to be
> through the pytables package. From my (admittedly cursory) reading it
> seems like pytables works with numarray rather than numeric.

As Robert said, PyTables works internally with numarray, but supports
Numeric by doing conversions. However the numarray --> Numeric (i.e.
for reading operations with Numeric flavor) conversions implies a
memory copy right now. Hopefully, this will be fixed sometime in the
future, with the advent of Numeric 24.0 or scipy.base.

Nevertheless, with the latest releases (those available in
http://www.carabos.com/downloads/pytables/snapshots/), I've included a
patch (based on a suggestion of Todd Miller) to optimize this
conversion, and with that, someone has reported the next input speeds:

""" 
On the SGI Altix with a Raid 5 disk array, up to 350 mb/s (using
Numeric).  With numarrays, I can approach 900 mb/s.

BTW, got a decent speed up on my Mac OS X machine as well, close to 50x.
"""

So, I think you can use PyTables with Numeric and still expect a
decent performance.

Cheers,

-- 
Francesc Altet