Congrats Francesc! On Tue, Jul 22, 2014 at 9:53 AM, Francesc Alted <faltet@gmail.com> wrote:
====================== Announcing bcolz 0.7.0 ======================
What's new ==========
In this release, support for Python 3 has been added, Pandas and HDF5/PyTables conversion, support for different compressors via latest release of Blosc, and a new `iterblocks()` iterator.
Also, intensive benchmarking has lead to an important tuning of buffer sizes parameters so that compression and evaluation goes faster than ever. Together, bcolz and the Blosc compressor, are finally fullfilling the promise of accelerating memory I/O, at least for some real scenarios:
http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/queryin...
``bcolz`` is a renaming of the ``carray`` project. The new goals for the project are to create simple, yet flexible compressed containers, that can live either on-disk or in-memory, and with some high-performance iterators (like `iter()`, `where()`) for querying them.
For more detailed info, see the release notes in: https://github.com/Blosc/bcolz/wiki/Release-Notes
What it is ==========
bcolz provides columnar and compressed data containers. Column storage allows for efficiently querying tables with a large number of columns. It also allows for cheap addition and removal of column. In addition, bcolz objects are compressed by default for reducing memory/disk I/O needs. The compression process is carried out internally by Blosc, a high-performance compressor that is optimized for binary data.
bcolz can use numexpr internally so as to accelerate many vector and query operations (although it can use pure NumPy for doing so too). numexpr optimizes the memory usage and use several cores for doing the computations, so it is blazing fast. Moreover, the carray/ctable containers can be disk-based, and it is possible to use them for seamlessly performing out-of-memory computations.
bcolz has minimal dependencies (NumPy), comes with an exhaustive test suite and fully supports both 32-bit and 64-bit platforms. Also, it is typically tested on both UNIX and Windows operating systems.
Installing ==========
bcolz is in the PyPI repository, so installing it is easy:
$ pip install -U bcolz
Resources =========
Visit the main bcolz site repository at: http://github.com/Blosc/bcolz
Manual: http://bcolz.blosc.org
Home of Blosc compressor: http://blosc.org
User's mail list: bcolz@googlegroups.com http://groups.google.com/group/bcolz
License is the new BSD: https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt
----
**Enjoy data!**
-- Francesc Alted _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion