zip() function troubles

Istvan Albert istvan.albert at gmail.com
Fri Jul 27 04:15:39 CEST 2007


On Jul 26, 9:33 pm, Paul Rubin <http://phr...@NOSPAM.invalid> wrote:

> Do a top or vmstat while that is happening and see if you are
> swapping.  You are allocating 10 million ints and 10 million tuple
> nodes, = 20 million objects.  Although, even at 100 bytes per object
> that would be 1GB which would fit in your machine easily.  Is it
> a 64 bit cpu?

we can safely drop the memory limit as being the cause and think about
something else

if you try it yourself  you'll see that it is very easy to generate 10
million tuples,
on my system it takes 3 (!!!) seconds to do the following:

size = 10**7
data = []
for i in range(10):
    x = [ (0,1) ] * size
    data.append( x )

Now it takes over two minutes to do this:

size = 10**7
a = [ 0 ] * size
b = zip(a,a)

the only explanation I can come up with is that the internal
implementation of zip must have some flaws





More information about the Python-list mailing list