[Python-Dev] genexps slow?

[Michael Hudson]

"Tim Peters" <tim.one at comcast.net> writes:

> [Tim Peters]
>>> Resuming a generator function is a lot cheaper than calling a
>>> function, but there's still a non-trivial amount of code to get in
>>> and out of eval_frame() each time, which the listcomp version gets
>>> to skip.
>
> [Michael Hudson.
>> This is something that occurred to me a while ago: how many opcodes
>> does a typical invocation of eval_frame actually execute?  A little
>> script told me that the median length of functions in Lib/*.py was 38
>> instructions (or 52 bytes) IIRC, but obviously a dynamic count is far
>> more interesting.  If the number is fairly small (and honestly, I have
>> no idea), the set up and tear down code becomes much more significant
>> than one might think.
>
> And whatever it is, I expect it will be significantly smaller for functions
> synthesized to implement generator expressions.

I expect invocations of most generators (not necessarily synthetic
genexp ones) probably execute *very* few instructions -- ~5, maybe?

>> I didn't think much about the code to get *out* of eval_frame.
>
> There's a twist there:  the functions synthesized for generator expressions
> get to take the quicker-than-general "fast_yield" exit path.  Hmm!  It looks
> like tests against WHY_YIELD before the fast_yield label can no longer
> succeed, but are still there to slow down non-generator cases.

Hmm.  I *think* I agree with you.  I also think the tests against
WHY_YIELD in the code for END_FINALLY are unnecessary, as we
(syntactically) don't allow yields when there are finally: statements
in play.  If I'm wrong about that, then you're wrong too :-) It's all
a bit subtle, to be sure.

Once again, I wonder if there was someway we could have two
eval_frames: one with tracing support, one without.  Unfortunately, I
don't think this can fly (how do you *start* tracing?).  Maybe we
could have a '-notrace' command line flag?  But I suspect that this is
a silly over-complication.

Cheers,
mwh

-- 
 "An infinite number of monkeys at an infinite number of keyboards
  could produce something like Usenet."
 "They could do a better job of it."
              -- the corollaries to Gene Spafford's Axiom #2 of Usenet