![](https://secure.gravatar.com/avatar/5c7407de6b47afcd3b3e2164ff5bcd45.jpg?s=120&d=mm&r=g)
PyTables is a library to deal with very large datasets. It leverages the excellent HDF5 and numarray libraries to allow doing that in a very efficient way using the Python language. More info in: http://pytables.sourceforge.net/ ========================= Announcing PyTables 1.2 ========================= The PyTables development team is happy to announce the availability of a new major version of PyTables package. This version sports a completely new in-memory tree implementation based around a *node cache system*. This system loads nodes only when needed and unloads them when they are rarely used. The new feature allows the opening and creation of HDF5 files with large hierarchies very quickly and with a low memory consumption (the object tree is no longer completely loaded in-memory), while retaining all the powerful browsing capabilities of the previous implementation of the object tree. You can read more about the dings and bells of the new cache system in: http://www.carabos.com/downloads/pytables/NewObjectTreeCache.pdf Also, Jeff Whitaker has kindly contributed a new module called tables.NetCDF. It is designed to be used as a drop-in replacement for Scientific.IO.NetCDF, with only minor actions to existing code. Also, if you have the Scientific.IO.NetCDF module installed, it allows to do conversions between HDF5 <--> NetCDF3 formats. Go to the PyTables web site for downloading the beast: http://pytables.sourceforge.net/ If you want more info about this release, please check out the more comprehensive announcement message available in: http://www.carabos.com/downloads/pytables/ANNOUNCE-1.2.html Acknowledgments =============== Thanks to the users who provided feature improvements, patches, bug reports, support and suggestions. See THANKS file in distribution package for a (incomplete) list of contributors. Many thanks also to SourceForge who have helped to make and distribute this package! And last but not least, a big thank you to THG (http://www.hdfgroup.org/) for sponsoring many of the new features recently introduced in PyTables. --- **Enjoy data!** -- The PyTables Team --
![](https://secure.gravatar.com/avatar/5dde29b54a3f1b76b2541d0a4a9b232c.jpg?s=120&d=mm&r=g)
Francesc Altet wrote:
What advantages does tables.NetCDF have over Scientific.IO.NetCDF? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
![](https://secure.gravatar.com/avatar/cdae659c5402abe203355fb7ed51bd0a.jpg?s=120&d=mm&r=g)
Chris Barker wrote:
Chris: From my perspective, the transparent compression (using zlib and the hdf5 shuffle filter) is the biggest one. I get files that are a factor of 2-4 smaller. -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@noaa.gov 325 Broadway Office : Skaggs Research Cntr 1D-124 Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
![](https://secure.gravatar.com/avatar/cdae659c5402abe203355fb7ed51bd0a.jpg?s=120&d=mm&r=g)
Francesc Altet wrote:
And any dimension can be 'unlimited' (not just the first). -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@noaa.gov 325 Broadway Office : Skaggs Research Cntr 1D-124 Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
![](https://secure.gravatar.com/avatar/5dde29b54a3f1b76b2541d0a4a9b232c.jpg?s=120&d=mm&r=g)
Francesc Altet wrote:
What advantages does tables.NetCDF have over Scientific.IO.NetCDF? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
![](https://secure.gravatar.com/avatar/cdae659c5402abe203355fb7ed51bd0a.jpg?s=120&d=mm&r=g)
Chris Barker wrote:
Chris: From my perspective, the transparent compression (using zlib and the hdf5 shuffle filter) is the biggest one. I get files that are a factor of 2-4 smaller. -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@noaa.gov 325 Broadway Office : Skaggs Research Cntr 1D-124 Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
![](https://secure.gravatar.com/avatar/cdae659c5402abe203355fb7ed51bd0a.jpg?s=120&d=mm&r=g)
Francesc Altet wrote:
And any dimension can be 'unlimited' (not just the first). -Jeff -- Jeffrey S. Whitaker Phone : (303)497-6313 Meteorologist FAX : (303)497-6449 NOAA/OAR/PSD R/PSD1 Email : Jeffrey.S.Whitaker@noaa.gov 325 Broadway Office : Skaggs Research Cntr 1D-124 Boulder, CO, USA 80303-3328 Web : http://tinyurl.com/5telg
participants (3)
-
Chris Barker
-
Francesc Altet
-
Jeff Whitaker