PEP 255: Simple Generators, Revised Posting

[Bernhard Herzog]

"Tim Peters" <tim.one at home.com> writes:

> [Bernhard Herzog]
> > With the current implementation that doesn't seem to be entirely true
> > (CVS from somtime 2001-06-23 afternoon UTC):
> >
> > >>> def empty():
> > ...     if 0: yield 0
> > ...
> > >>> for i in empty():
> > ...     print i
> > ...
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in ?
> > TypeError: iter() of non-sequence
> > >>> import dis
> > >>> dis.dis(empty)
> >           0 SET_LINENO               1
> >
> >           3 SET_LINENO               2
> >           6 LOAD_CONST               0 (None)
> >           9 RETURN_VALUE
> > >>> empty.func_code.co_flags
> > 3
> >
> > It seems that the mere lexical presence of the yield statement doesn't
> > make a function a generator. The compiler apparently optimized the if
> > statement away before testing whether it's a generator.
> 
> Yes, it does.  So don't do that.  This is a general glitchlet, i.e. not specific
> to generators; e.g.
> 
> >>> k = 12
> >>> def f():
> ...     if 0:
> ...         k = 3
> ...     print k
> ...
> >>> f()
> 12
> >>>
> 
> That didn't yield "the expected" UnboundLocalError for the same reason

I wasn't aware of that! For that matter, I wasn't even aware that the
compiler removed "if 0" blocks. When was that introduced. It even
"works" with 1.5.2.

Anyway, I tried the example I gave because I wanted to see how to create
an empty generator, i.e. the generator equivalent of an empty sequence.

For a normal function it's simply

    def f():
        pass

or

    def f():
        return


but for a generator that obviously doesn't work; you need to introduce a
yield statement and make sure that it's never executed. The following
works:

>>> def g():
..     if ():
..             yield 0
.. 
>>> for i in g():
..     print i
.. 
>>> 

The compiler apparently isn't smart enough to think of () as a constant
(the bytecode confirms that).

Being able to easily write a function that does nothing (in fact it's
the most trivial function one can write) is very useful in practice, so
as a generalization being able to write empty generators might be
useful, too. I'm not sure about that, though, because I don't have
enough experience with them. 

It seems to me that as it stands, empty generators are a bit too
difficult to write, so I think we need an easier way to write them. One
way to write them would be

    def g():
        yield

i.e. with a parameterless yield statement. Come to think of it, in
generators, a yield without an argument could be equivalent to return.

If generators weren't created with a def statement it would be a bit
more obvious:

   generator g():
        pass


> affected blocks are actually removed by the *parser*, and "the
> compiler" proper never knows they were there -- the "if 0" blocks
> aren't even in the parse tree. That makes it difficult to repair, and
> since nobody writes "if 0" in real life, that in turn makes the
> cost/benefit ratio very high.

In hand written code, "if 0" is indeed a bit unlikely, although I have
sometimes changed the expression in an if statement to 0 for debugging
purposes and if that can change the compiler's idea of local variables
and other static aspects of the code it could lead to unexpected
behavior. I've never run into this, though, so it may not be much of a
problem in practice.

   Bernhard


-- 
Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                               http://mapit.de/