[Numpy-discussion] ANN: carray 0.3 released

Kevin Jacobs <jacobs@bioinformed.com> bioinformed at gmail.com
Thu Dec 23 23:34:07 EST 2010

On Wed, Dec 22, 2010 at 1:58 PM, Francesc Alted <faltet at pytables.org> wrote:

>  >>> %time b = ca.zeros(1e12)
>  CPU times: user 54.76 s, sys: 0.03 s, total: 54.79 s
>  Wall time: 55.23 s

I know this is somewhat missing the point of your demonstration, but 55
seconds to create an empty 3 GB data structure to represent a multi-TB dense
array doesn't seem all that fast to me.  Compression can do a lot of things,
but isn't this a case where a true sparse data structure would be the right
tool for the job?  I'm more interested in seeing what a carray can do with
census data, web logs, or somethat vaguely real world where direct binary
representations are used by default and assumed to be reasonable optimal
(i.e., anything sensibly stored in sqlite tables).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20101223/d53aa1cf/attachment.html>

More information about the NumPy-Discussion mailing list