Loop performance disappearance

[Chris Ryland]

Several people kindly pointed out that it's hard in Python, because the
definition of "range" might be anything.

I guess xrange is an OK compromise, but this points up the need (observed in
other threads recently) for some kind of optimizing compiler that will take
a whole project, do dataflow and definitional analysis on it, perhaps take
your word for it that you're not replacing any of the built-in functions or
classes in an exec/eval, and do some serious optimizations along these
lines.

If Python perdures and matures for the next N years (as it gives every
evidence of doing), then I suppose this will come in time.

--
Cheers!
/ Chris Ryland, President / Em Software, Inc. / www.emsoftware.com
"Chris Ryland" <cpr at emsoftware.com> wrote in message
news:FCRz4.25$6k3.1016 at news...
> I always assumed (perhaps being too much of a newbie) that
>
> for i in range(n):
>    :
>
> would automatically be recognized by the compiler and turned into a simple
> loop control using i alone (like turning it into an xrange, I guess).
Seems
> simple enough and common enough that it should be automatic, no? (Forgive
> any massive display of ignorance here.)
>
> --
> Cheers!
> / Chris Ryland, President / Em Software, Inc. / www.emsoftware.com
> "Remco Gerlich" <scarblac-spamtrap at pino.selwerd.nl> wrote in message
> news:slrn8cva9a.aj8.scarblac-spamtrap at flits104-37.flits.rug.nl...
> > Mikael Johansson wrote in comp.lang.python:
> > > I was just wondering what the reason for the huge performance decrease
> > > in a loop execution when the number of steps exceeds some critical
value
> > > is. To give an example:
> > >
> > > for i in range(loops):
> > >     pass
> > >
> > > If loops=500 000 (space added for clarity) the "program" executes in
~2
> > > secs on my machine. But if loops is set to 5 000 000, the execution
time
> > > rises to substantially more than tenfold (I terminated it after one
> > > minute). However the CPU-load is quite small, most of the time is
spent
> > > disk swapping like crazy!
> >
> > That's because range(loops) is a list. It actually has to construct a
list
> > that size, and then walks through it. 5 million doesn't fit in your
memory
> > and it starts swapping.
> >
> > > Any ideas of the reason for this, work-arounds?
> >
> > Use xrange instead of range. xrange doesn't return a list, but an object
> > that generates values on demand, simulating a list. Slightly slower, but
> > doesn't need the memory.
> >
> > --
> > Remco Gerlich,  scarblac at pino.selwerd.nl
> >   Murphy's Rules, "Which is why you get 'em so cheap":
> >    In SPI's Universe, the sword is prohibited from use at any combat
> >    range.
>
>