PyTables 2.2 released: entering the multi-core age
Francesc Alted
faltet at pytables.org
Thu Jul 1 21:19:07 CEST 2010
=================================
Announcing PyTables 2.2 (final)
=================================
I'm happy to announce PyTables 2.2 (final). After 18 months of
continuous development and testing, this is, by far, the most powerful
and well-tested release ever. I hope you like it too.
What's new
==========
The main new features in 2.2 series are:
* A new compressor called Blosc, designed to read/write data to/from
memory at speeds that can be faster than a system `memcpy()` call.
With it, many internal PyTables operations that are currently
bounded by CPU or I/O bandwith are speed-up. Some benchmarks:
http://blosc.pytables.org/trac/wiki/SyntheticBenchmarks
And a demonstration on how Blosc can improve PyTables performance:
http://www.pytables.org/docs/manual/ch05.html#chunksizeFineTune
* Support for HDF5 hard links, soft links and external links (kind of
mounting external filesystems). A new tutorial about its usage has
been added to the 'Tutorials' chapter of User's Manual. See:
http://www.pytables.org/docs/manual/ch03.html#LinksTutorial
* A new `tables.Expr` module (based on Numexpr) that allows to do
persistent, on-disk computations on many algebraic operations.
For a brief look on its performance, see:
http://pytables.org/moin/ComputingKernel
* Suport for 'fancy' indexing (i.e., à la NumPy) in all the data
containers in PyTables. Backported from the implementation in the
h5py project. Thanks to Andrew Collette for his fine work on this!
* Binaries for both Windows 32-bit and 64-bit are provided now.
As always, a large amount of bugs have been addressed and squashed too.
In case you want to know more in detail what has changed in this
version, have a look at:
http://www.pytables.org/moin/ReleaseNotes/Release_2.2
You can download a source package with generated PDF and HTML docs, as
well as binaries for Windows, from:
http://www.pytables.org/download/stable
For an on-line version of the manual, visit:
http://www.pytables.org/docs/manual-2.2
What it is?
===========
PyTables is a library for managing hierarchical datasets and designed to
efficiently cope with extremely large amounts of data with support for
full 64-bit file addressing. PyTables runs on top of the HDF5 library
and NumPy package for achieving maximum throughput and convenient use.
Resources
=========
About PyTables:
http://www.pytables.org
About the HDF5 library:
http://hdfgroup.org/HDF5/
About NumPy:
http://numpy.scipy.org/
Acknowledgments
===============
Thanks to many users who provided feature improvements, patches, bug
reports, support and suggestions. See the ``THANKS`` file in the
distribution package for a (incomplete) list of contributors. Most
specially, a lot of kudos go to the HDF5 and NumPy (and numarray!)
makers. Without them, PyTables simply would not exist.
Share your experience
=====================
Let us know of any bugs, suggestions, gripes, kudos, etc. you may
have.
----
**Enjoy data!**
-- The PyTables Team
--
Francesc Alted
More information about the Python-announce-list
mailing list