Simple question about how the optimizer works

[James J. Besemer]

Jeff Epler wrote:

> On Thu, May 09, 2002 at 08:52:28AM -0700, James J. Besemer wrote:
>
> > I'd expect there'd be more to gain in Python simply looking for patterns such as
> >
> >     for i in xrange(N): ...
>
> The expression giving the object to iterate over in a 'for' loop is
> evaluated only when the 'for' loop begins, not at each iteration.

No, of course not.  I didn't mean to suggest that the expression in this case was
being evaluated multiple times.

Rather I was harkening back to an earlier discussion where we established that certain
forms of for loops and list comprehensions were much slower than e.g., functional and
lambda forms that accomplished the same things.

So I was suggesting an optimizer that when it saw a form like:

    res = []
    for x in list:
        if x:
            res.append( x )

it would substitute a call to something

    filter( lambda x : x is not None, list )

Where the entire substitution might furthermore be part of a C runtime routine instead
even of explicit calls to filter and lambda.

List comprehensions seem to be one of the slowest way to iterate over data.  Perhaps
some common forms can be recognized by the optimizer and passed to an efficient
runtime module.

> Get busy ..

Oh yeah.  Lotta work.

I haven't looked at Python's runtime system but optimizations was an area I was
interested in.

Regards

--jb

--
James J. Besemer  503-280-0838 voice
http://cascade-sys.com  503-280-0375 fax
mailto:jb at cascade-sys.com