[Python-ideas] [Python-Dev] yield * (Re: Missing operator.call)

[Steven D'Aprano]

Antoine Pitrou wrote:
> Steven D'Aprano <steve at ...> writes:
>> Marcin 'Qrczak' Kowalczyk wrote:
>>> On Sun, Feb 8, 2009 at 23:20, Steven D'Aprano <steve at ...> wrote:
>>>
>>>> And the cost is small:
>>>>
>>>> [steve <at> ando ~]$ python -m timeit -s "seq = range(500)" "(3*x for x in
> seq if
>>>> x%2 == 1)"
>>>> 1000000 loops, best of 3: 0.611 usec per loop
>>> Because generators are lazy and you don't run it into completion.
>> We were talking about the cost of *making* the generator, not the cost 
>> of running it to completion.
> 
> No, I was talking about the cost of running it to completion. 

Perhaps I misunderstood you, because what you actually said was:

"But the former will be slower than the latter, because it constructs an
intermediate generator only to yield it element by element."

Since a for loop will also yield element by element, the only difference 
I saw was constructing the generator expression, which is cheap.

 > A generator is
> executed in a separated frame. Therefore, if you "yield from" a generator, there
> is a frame switch at each iteration between the generator frame and the frame of
> the "yield from".
> Which is not the case with an inline "for" loop containing a "yield".

I'm afraid I don't understand the relevance of this. If it's a 
criticism, it's a criticism of generators in general, not of the 
proposed syntax. Don't we already carry the cost of the frame switch 
when iterating over a generator?

for el in generator:
     yield el

If that is replaced with the proposed syntax

yield from generator

what's the difference, performance-wise? In both cases, you can optimise 
by unrolling the generator into an inline for loop, at the cost of 
readability, convenience, and the ability to pass generator objects around.

In Python 2.4 at least, the optimisation is not to be sneered at (modulo 
the usual warnings about premature optimisation): unrolling is about 
30-40% faster:

$ python -m timeit -s "def f():" -s "    for x in (i+1 for i in 
xrange(20)): yield x" "list(f())"  # using gen expr
100000 loops, best of 3: 12.9 usec per loop

$ python -m timeit -s "def f():" -s "    for x in xrange(20): yield x+1" 
"list(f())"  # unrolled into body of the loop
100000 loops, best of 3: 8.09 usec per loop


Since people are already choosing to use generator expressions instead 
of unrolling them into for loops, I don't believe that your objection is 
relevant to the proposal. "yield from expression" would (presumably) be 
a shorter, neater way of saying "for x in expression: yield x" except 
that it doesn't create a new name x.



-- 
Steven