[code-quality] Warn on list comprehension variables used outside list comprehensions?

[Keith Derrick]

I couldn't find anything in their bug list about this, so thought I'd check.

Just ran pylint at the command line with pylint 1.4 on this file 
(docstrings and import just to suppress miscellaneous complaints)

>      1    ''' Test file for pylint '''
>      2    from __future__ import print_function
>      3
>      4    def method1():
>      5        ''' Let x bleed from comprehension '''
>      6        [x for x in range(3)]
>      7        print(x)
>      8
>      9    def method2():
>     10        ''' reuse/hide x from local scope in comprehension '''
>     11        y = 10
>     12        [y for y in range(3)]
>     13        print(y)
>     14

For python 2.7.6
> ************* Module bug
> W:  6, 4: Expression "[x for x in range(3)]" is assigned to nothing 
> (expression-not-assigned)
> W:  7,10: Using possibly undefined loop variable 'x' 
> (undefined-loop-variable)
> C: 11, 4: Invalid variable name "y" (invalid-name)
> W: 12, 4: Expression "[y for y in range(3)]" is assigned to nothing 
> (expression-not-assigned)
For python 3.4.0
> ************* Module bug
> W:  6, 4: Expression "[x for x in range(3)]" is assigned to nothing 
> (expression-not-assigned)
> E:  7,10: Undefined variable 'x' (undefined-variable)
> C: 11, 4: Invalid variable name "y" (invalid-name)
> W: 12, 4: Expression "[y for y in range(3)]" is assigned to nothing 
> (expression-not-assigned)

So, it warns about the first case in py2 and gives an error in py3 which 
is expected.

But it seems to be confused by the second case, flagging "y" as an 
invalid variable-name at the assignment point for both versions.

Keith Derrick | Principal Engineer, Connected Platform | Engineering
LG Silicon Valley Lab | 5150 Gt America Parkway, Santa Clara, CA 95054
Office: 408.610-5746 | Mobile: 831.383.9567 |  LG.com

On 11/26/2014 07:50 AM, Ian Cordasco wrote:
> On Wed, Nov 26, 2014 at 9:27 AM, Keith Derrick <keith.derrick at lge.com> wrote:
>> Definitely seems like something that should be warned about.
>>
>> For python2 it ought to be an error (which can always be turned off) for
>> either code snippet.
> PyFlakes doesn't allow you to ignore errors. Flake8 does so this is a
> moot point.
>
>> For python3, I'd say the two line snippet should flag an error - yes, it
>> will fail at run-time, but pydev in eclipse runs pylint and saves me a
>> lot of grief by spotting this type of thing before I ever save the file
>> (though it doesn't catch either of these, presumably because they are
>> valid python).
> If PyLint doesn't catch this I wonder if it's either never been raised
> or been decided to be a bad idea. Could someone search their bugs?
>
>> The three line snippet should probably give a warning ("Are you sure you
>> mean this?")
>>
>> The question is, how do we know which version of Python the code under
>> inspection is targeted for, and hence how to treat this case.
>>
>> Some thoughts on that:
>>
>> * Anyone serious enough to be using pyflakes, pylint, or other static
>> analysis tools is also probably using virtualenv.
>> * pyflakes is therefore most likely running under the same python
>> version for which the code is targeted.
>> * code which is meant to run under multiple python versions is probably
>> tested under something like tox - so again, pyflakes is run under the
>> targeted python version (certainly my tox.ini runs it in every target
>> environment)
>>
>> There's a lot of "probably"'s in there, but I don't think they are
>> unreasonable assumptions.
>>
>> So, could  pyflakes reasonably assume that the interpreter under which
>> it is running is also the intended target version for the code under
>> inspection? At least for the major version.
> It already works this way. If you use pyflakes on Python 2 but are
> writing code only valid in Python 3 you will have a bad time. You have
> to install pyflakes on the version of Python you want it to lint for
> since pyflakes works only with the in-built ast module of Python.
>
>> This could then be overridden by a suitable command line option to
>> specify the target version(s)
> No. This isn't how PyFlakes is intended to work and this is extraneous.
>
>> Also, in cases like this, possibly a "This will behave differently in
>> Python 2 and 3" warning is a worth while alternative.
> Again, PyFlakes is very different from PyLint. We don't issue warnings
> like this.
>
>> Keith Derrick | Principal Engineer, Connected Platform | Engineering
>> LG Silicon Valley Lab | 5150 Gt America Parkway, Santa Clara, CA 95054
>> Office: 408.610-5746 | Mobile: 831.383.9567 |  LG.com
>>
>> On 11/26/2014 06:57 AM, Skip Montanaro wrote:
>>>> That's not the point of this check. The point of this check is that on
>>>> Python 2, the binding to x in the comprehension bleeds outside of the
>>>> comprehension scope
>>> Got it.
>>>
>>> Still, this code:
>>>
>>>>       x = 10
>>>>       [x for x in range(3)]
>>>>       print(x + 1)
>>> will run differently in Python 2 than Python3, so even if that was a
>>> conscious choice by the author, a --py3k flag should cause a message
>>> for this code. This is even worse than the simpler
>>>
>>>>       [x for x in range(3)]
>>>>       print(x + 1)
>>> because at least that will raise a NameError when run in Python 3. The
>>> three-line construct will still run, though produce different output.
>>>
>>> Skip
>>> _______________________________________________
>>> code-quality mailing list
>>> code-quality at python.org
>>> https://mail.python.org/mailman/listinfo/code-quality
>> _______________________________________________
>> code-quality mailing list
>> code-quality at python.org
>> https://mail.python.org/mailman/listinfo/code-quality