[Tutor] Memory error for list creation

[Wayne Werner]

On Tue, Aug 24, 2010 at 9:54 AM, Triantafyllos Gkikopoulos <
T.Gkikopoulos at dundee.ac.uk> wrote:

>  Hi,
>
>  I am looking for an alternative to:
>
>
> ************************************************************
> Please consider the environment. Do you really need to print this email?
>
>
>
> >>> listx=[[[] for k in range(ds)] for j in range(i)]
>
> as right now I am getting a Memory error on this, I tried this also on a
> cluster node with something like 16GB of memory and it didn't solve the
> problem.
>
>

Unless you're using Python 3.x, use xrange - because here you're creating i
copies of the list 0..ds-1.

If you're trying to create a matrix this isn't the most effective way to get
it, but if you want a jagged array then using xrange should help your memory
consumption.


> listx is subsequently used:
>
> >>> for x in something:
> >>>      for y in x:
> >>>                   listx[ii][y[1]].append(y[0])
> >>>      ii+=1
>
>
>
>  For reference values for k range from 300 -1200 and for i ~5000.
>
>
> I though about using scipy/numpy array but then I wouldn't be able to have
> the flexibility of using append or not having to predefine the size.
>

If you need the ability that bad, you can convert the array to a list and
vice versa - of course those operations take time, so YMMV.

HTH,
Wayne
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20100824/19aba8d5/attachment.html>