Some simple performace tests (long)

[Terry Reedy]

"TPJ" <tprimke at interia.pl> wrote in message 
news:1123334835.548075.129430 at z14g2000cwz.googlegroups.com...
> "The advantage of xrange() over range() is minimal (since xrange()
> still has to create the values when asked for them) except when a very
> large range is used on a memory-starved machine or when all of the
> range's elements are never used (such as when the loop is usually
> terminated with break)." - from Python Library Reference.

Nothing in your results, properly interpreted, contradicts this.

> I decided to measure the performance of range and xrange.

On one machine, with one binary.  Relative performance on different systems 
can vary by, say, 10%.  But of course, if you are optimizing a program to 
run on just one machine, then the results on that machine are all that 
matters.

> I did it with the following functions:
>
> def rprint( n ):
>  a = time.time()
>  for i in range(n): print i
>  print time.time() - a
>
> def xrprint( n ):
>  a = time.time()
>  for i in xrange(n): print i
>  print time.time() - a

To compare two things, one wants to remove all possible noise factors.  If 
one wants relative performance (ratios, percentages) then constant factors 
should also be removed, when possible, to make the apparent difference as 
close to the real difference as possible.   Print is a large constant + 
large noise factor that you *DO NOT* want for the stated comparison.  It 
requires a call through lots of OS code with variable execution times.

> def rpass( n ):
>  a = time.time()
>  for i in range(n): pass
>  print time.time() - a
>
> def xrpass( n ):
>  a = time.time()
>  for i in xrange(n): pass
>  print time.time() - a

This is the right comparison for the reasons noted.

> The results were as follows:
>
> n       rprint          xrprint
>
> 10^4    0.37 s          0.34 s          <- (1)
> 10^5    4.26 s          4.25 s
> 10^6    42.57 s         42.57 s
> 10^7    431.94 s        438.32 s        <- (2)

These times with print are about 300x the results below without print. 
They are useless for comparing range and xrange.  The differences above are 
differences in printing (display) times and not in range versus xrange.

Think about it.  When the differences between runs with print are several 
times as large as the total time without, then those large differences 
cannot have anything to do with the minor difference in looping method.

> n       rpass           xpass
>
> 10^4    0.0012 s        0.0011 s
> 10^5    0.0220 s        0.0139 s
> 10^6    0.1463 s        0.1298 s
> 10^7    1.4818 s        1.1807 s

The looks more coherent.

> The values are the average times printed by tested functions.
>
> Conclusions:
>
> 1) According to (1) I could say that xrange might be somewhat faster
> than range with lower numbers of iterations.
> 2) According to (2) I could say that xrange might be slower than range
> with higher number of iterations.

You could, but you would be wrong.  The second table shows that xrange is 
slightly faster on your machine over the range tested.

> The test with pass is not so important as the test with print (because
> we usually do something inside of loops).

This is completely backwards for the reasons already given.  The point 
about doing something inside loops in relevant in so far as it says the 
that the minor difference between range and xrange, whichever way it goes 
on a particular system, will matter relatively even less in application. 
So the choice hardly matters unless space is a considerations.  Which is 
what the quoted docs more or less said.

> import array, random, time

To reduce random noise from random, the generator should be reinitialized 
with the seed(someint) at the beginning of each function.  This is the 
purpose of seed(x).  But for the comparisons you are making, randomness is 
irrelevance and the time for calls to random a nuisance.

> def test1( n, size ):
>  a = time.time()
>  for i in xrange(n):
>    l = []
>    for i in xrange(size):
>      l.append( random.randint( 1,10 ) )
>    e = sum(l) / float(size)  # just for taking some time

This is exactly what you DO NOT WANT TO DO for comparing anything else.

>  print time.time() - a

> def test2( n, size ):
>  a = time.time()
>  l = []
>  for i in xrange(n):
>    del l[:]
>    for i in xrange(size):
>      l.append( random.randint( 1,10 ) )
>    e = sum(l) / float(size)
>  print time.time() - a

This is trivially different.  To test 'l = []' versus 'del l[:]', test just 
that.

> def test3( n, size ):
>  a = time.time()
>  l = range(size)
>  for i in xrange(n):
>    for i in xrange(size):
>      l[i] = random.randint( 1,10 )
>    e = sum(l) / float(size)
>  print time.time() - a

It is not surprising that allocating a list just once and filling it in is 
faster than starting empty and expanding and copying multiple times.  In 
any case, if you want to test that, test just that without all the noise 
and masking of other stuff.

I did not look at the slicing stuff.

Terry J. Reedy