extremely slow array indexing?

Will McGugan will at willmcgugan.com
Thu Nov 30 12:48:43 EST 2006


Grace Fang wrote:

> Hi,
>
> I am writing code to sort the columns according to the sum of each
> column. The dataset is huge (50k rows x 300k cols), so i need to read
> line by line and do the summation to avoid the out-of-memory problem.
> But I don't know why it runs very slow, and part of the code is as
> follows. I suspect it's because of array index, but not sure. Can
> anyone
> point out what needs to be modified to make it run fast? thanks in
> advance!

Array indexing is unlikely to be the culprit. Could it not just be
slow, because you are processing a lot of data? With numbers those big
I would expect to have enough time to go make a coffee, then drink it.

If you think it is slower than it could be, post more code for
optimization advice...

Will McGugan
--
http://www.willmcgugan.com




More information about the Python-list mailing list