[Python-ideas] Making it easy to prepare for PEP479

[Chris Angelico]

On Mon, May 18, 2015 at 6:14 PM, Ram Rachum <ram at rachum.com> wrote:
> Hi everybody,
>
> I just heard about PEP479, and I want to prepare my open-source projects for
> it.
>
> I have no problem changing the code so it won't depend on StopIteration to
> stop generators, but I'd also like to test it in my test suite. In Python
> 3.5 I could use `from __future__ import generator_stop` so the test would be
> real (i.e. would fail wherever I rely on StopIteration to stop a generator).
> But I can't really put this snippet in my code because then it would fail on
> all Python versions below 3.5.
>
> This makes me think of two ideas:
>
> 1. Maybe we should allow `from __future__ import whatever` in code, even if
> `whatever` wasn't invented yet, and simply make it a no-op? This wouldn't
> help now but it could prevent these problems in the future.

Downside: A typo would silently stop a future directive from working.
If "from __future__ import generator_stop" doesn't cause an error in
<3.5, then "from __future__ import genarator_stop" would cause no
error in any version, and that's a problem.

> 2. Maybe introduce a way to do `from __future__ import generator_stop`
> without including it in code? Maybe a flag to the `python` command? (If
> something like this exists please let me know.)

The problem is that it's hard to try-except special directives like
this. You can try-except a regular import, catch the run-time error,
and do something else; but short of exec'ing your code, you can't
catch SyntaxError. I'm not sure how best to deal with this.

However, it ought to be possible to simply run your tests with
generator_stop active, even if that means using exec instead of
regular imports. Something like this:

# utils.py
# In the presence of generator_stop, this will bomb

def f():
    raise StopIteration

def g():
    yield f()


# test_utils.py

# Instead of:
# import utils
# Try this:
with open("utils.py") as f:
    code = "from __future__ import generator_stop\n" + f.read()
import sys # Any module at all
utils = type(sys)("utils")
exec(code,vars(utils))

# At this point, you can write regular tests involving the
# 'utils' module, which has been executed in the presence
# of the generator_stop directive.
list(utils.g())

It's ugly, and it depends on the module being in the current directory
(though you could probably use importlib to deal with that part), but
it's just for your tests. I don't know of any way to simplify this
out, but it may well be possible (using some mechanism similar to what
the interactive interpreter does); in any case, all the ugliness
should be in a single block up the top of your test runner - and you
could turn it into a function, as you'll probably want to do this for
lots of modules.

Experts of python-ideas, is there a way to use an import hook to do this?

ChrisA