[Numpy-discussion] [ANN] bcolz 0.7.2
valentin at haenel.co
Sun Oct 12 12:40:33 EDT 2014
Announcing bcolz 0.7.2
This is a maintenance release that fixes various bits and pieces.
Importantly, compatibility with Numpy 1.9 and Cython 0.21 has been fixed
and the test suit no longer segfaults on 32 bit UNIX. Feature-wise a new
``carray.view()`` method has been introduced which allows carrays to
share the same raw data.
``bcolz`` is a renaming of the ``carray`` project. The new goals for
the project are to create simple, yet flexible compressed containers,
that can live either on-disk or in-memory, and with some
high-performance iterators (like `iter()`, `where()`) for querying them.
Together, bcolz and the Blosc compressor, are finally fulfilling the
promise of accelerating memory I/O, at least for some real scenarios:
For more detailed info, see the release notes in:
What it is
bcolz provides columnar and compressed data containers. Column storage
allows for efficiently querying tables with a large number of columns.
It also allows for cheap addition and removal of column. In addition,
bcolz objects are compressed by default for reducing memory/disk I/O
needs. The compression process is carried out internally by Blosc, a
high-performance compressor that is optimized for binary data.
bcolz can use numexpr internally so as to accelerate many vector and
query operations (although it can use pure NumPy for doing so too).
numexpr optimizes the memory usage and use several cores for doing the
computations, so it is blazing fast. Moreover, the carray/ctable
containers can be disk-based, and it is possible to use them for
seamlessly performing out-of-memory computations.
bcolz has minimal dependencies (NumPy), comes with an exhaustive test
suite and fully supports both 32-bit and 64-bit platforms. Also, it is
typically tested on both UNIX and Windows operating systems.
bcolz is in the PyPI repository, so installing it is easy::
$ pip install -U bcolz
Visit the main bcolz site repository at:
Home of Blosc compressor:
User's mail list:
bcolz at googlegroups.com
License is the new BSD:
More information about the NumPy-Discussion