[Python-ideas] Making it easy to prepare for PEP479

[Steven D'Aprano]

On Tue, May 19, 2015 at 03:46:49PM +1000, Chris Angelico wrote:

> > The only solution I can come up with is to use the inspect module to
> > fetch the generator's source code and scan it for "raise StopIteration".
> > Parsing the AST will also work, or even the byte-code at a pinch, but
> > the source code is easiest:
> >
> > assertFalse("raise StopIteration" in source)
> >
> > That will fail if the generator raises StopIteration directly regardless
> > of version. It doesn't catch *all possible* violations, e.g.:
> 
> More significant example: It doesn't catch a codebase that has some
> functions which are used in generators and others which are used in
> class-based iterators.

Obviously I only sketched a solution. The person writing the tests has 
to distinguish between functions or methods which must not call "raise 
StopIteration", and test them, while avoiding testing those which may 
use raise. They may want to test more than just the generator function 
themselves, e.g. any functions they call.

In principle, if you're reading the code or the AST, you can do a static 
analysis to automatically detect what functions it calls, and scan them 
as well, but that's a lot of effort for mere unit tests, and the chances 
are that your test code will be buggier than your non-test code. Easier 
to just add the called functions to a list of functions to be checked.

The person writing the tests must decide how much he cares about this. 
"Do the simplest thing that can possibly work" applies to tests as well 
as code (tests *are* code).

(In my opinion, just by *reading* this thread, Ram has already exceeded 
the amount of time and energy that these tests are worth.)


> > So, I believe that the whole __future__ directive is a red herring, and
> > doesn't actually help Ram do what he wants, which is to write tests
> > which will fail if his generators call raise StopIteration regardless of
> > what version of Python he runs the test under.
> 
> Okay. So how do you ensure that Python 3.7 and Python 3.4 can both run
> your code?

If I am right that the future directive is irrelevant, then you simply 
*don't include the future directive*.

Or you split the code into parts that don't require the directive, and 
parts that do, and put them in different files, then conditionally 
import the second set, either in a try...except or if version... block.

Or you write one file: test.py, and run your tests with a wrapper 
script which duplicates that file and inserts the future directive:

# untested
cp test.py test479.py
sed -i '1i from __future__ import feature' test479.py
python -m unittest test.py
python -m unittest test479.py


Combine and adjust as needed.


-- 
Steve