[SciPy-User] [Numpy-discussion] ANN: carray 0.3 released

[Kevin Jacobs <jacobs@bioinformed.com>]

On Wed, Dec 22, 2010 at 1:58 PM, Francesc Alted <faltet at pytables.org> wrote:

>  >>> %time b = ca.zeros(1e12)
>  CPU times: user 54.76 s, sys: 0.03 s, total: 54.79 s
>  Wall time: 55.23 s
>

I know this is somewhat missing the point of your demonstration, but 55
seconds to create an empty 3 GB data structure to represent a multi-TB dense
array doesn't seem all that fast to me.  Compression can do a lot of things,
but isn't this a case where a true sparse data structure would be the right
tool for the job?  I'm more interested in seeing what a carray can do with
census data, web logs, or somethat vaguely real world where direct binary
representations are used by default and assumed to be reasonable optimal
(i.e., anything sensibly stored in sqlite tables).

-Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20101223/d53aa1cf/attachment.html>