Mailman 3 PyLint catching comprehension binding leaks (was: Warn on list comprehension variables used outside list comprehensions?) - code-quality

26 Nov 2014


      On Wed, Nov 26, 2014 at 10:51 AM, Keith Derrick  wrote:
...
I couldn't find anything in their bug list about this, so thought I'd check.
Just ran pylint at the command line with pylint 1.4 on this file
(docstrings and import just to suppress miscellaneous complaints)
...
1    ''' Test file for pylint '''
     2    from __future__ import print_function
     3
     4    def method1():
     5        ''' Let x bleed from comprehension '''
     6        [x for x in range(3)]
     7        print(x)
     8
     9    def method2():
    10        ''' reuse/hide x from local scope in comprehension '''
    11        y = 10
    12        [y for y in range(3)]
    13        print(y)
    14
For python 2.7.6
...
************* Module bug
W:  6, 4: Expression "[x for x in range(3)]" is assigned to nothing
(expression-not-assigned)
W:  7,10: Using possibly undefined loop variable 'x'
(undefined-loop-variable)
C: 11, 4: Invalid variable name "y" (invalid-name)
W: 12, 4: Expression "[y for y in range(3)]" is assigned to nothing
(expression-not-assigned)
For python 3.4.0
************* Module bug
W:  6, 4: Expression "[x for x in range(3)]" is assigned to nothing
(expression-not-assigned)
E:  7,10: Undefined variable 'x' (undefined-variable)
C: 11, 4: Invalid variable name "y" (invalid-name)
W: 12, 4: Expression "[y for y in range(3)]" is assigned to nothing
(expression-not-assigned)
So, it warns about the first case in py2 and gives an error in py3 which
is expected.
But it seems to be confused by the second case, flagging "y" as an
invalid variable-name at the assignment point for both versions.
Keith Derrick | Principal Engineer, Connected Platform | Engineering
LG Silicon Valley Lab | 5150 Gt America Parkway, Santa Clara, CA 95054
Office: 408.610-5746 | Mobile: 831.383.9567 |  LG.com
At this point, I'd recommend repling to the retitled message to catch
the PyLint maintainer's focus.
...
On 11/26/2014 07:50 AM, Ian Cordasco wrote:
...
On Wed, Nov 26, 2014 at 9:27 AM, Keith Derrick  wrote:
...
Definitely seems like something that should be warned about.
For python2 it ought to be an error (which can always be turned off) for
either code snippet.
PyFlakes doesn't allow you to ignore errors. Flake8 does so this is a
moot point.
...
For python3, I'd say the two line snippet should flag an error - yes, it
will fail at run-time, but pydev in eclipse runs pylint and saves me a
lot of grief by spotting this type of thing before I ever save the file
(though it doesn't catch either of these, presumably because they are
valid python).
If PyLint doesn't catch this I wonder if it's either never been raised
or been decided to be a bad idea. Could someone search their bugs?
...
The three line snippet should probably give a warning ("Are you sure you
mean this?")
The question is, how do we know which version of Python the code under
inspection is targeted for, and hence how to treat this case.
Some thoughts on that:
* Anyone serious enough to be using pyflakes, pylint, or other static
analysis tools is also probably using virtualenv.
* pyflakes is therefore most likely running under the same python
version for which the code is targeted.
* code which is meant to run under multiple python versions is probably
tested under something like tox - so again, pyflakes is run under the
targeted python version (certainly my tox.ini runs it in every target
environment)
There's a lot of "probably"'s in there, but I don't think they are
unreasonable assumptions.
So, could  pyflakes reasonably assume that the interpreter under which
it is running is also the intended target version for the code under
inspection? At least for the major version.
It already works this way. If you use pyflakes on Python 2 but are
writing code only valid in Python 3 you will have a bad time. You have
to install pyflakes on the version of Python you want it to lint for
since pyflakes works only with the in-built ast module of Python.
...
This could then be overridden by a suitable command line option to
specify the target version(s)
No. This isn't how PyFlakes is intended to work and this is extraneous.
...
Also, in cases like this, possibly a "This will behave differently in
Python 2 and 3" warning is a worth while alternative.
Again, PyFlakes is very different from PyLint. We don't issue warnings
like this.
...
Keith Derrick | Principal Engineer, Connected Platform | Engineering
LG Silicon Valley Lab | 5150 Gt America Parkway, Santa Clara, CA 95054
Office: 408.610-5746 | Mobile: 831.383.9567 |  LG.com
On 11/26/2014 06:57 AM, Skip Montanaro wrote:
...
...
That's not the point of this check. The point of this check is that on
Python 2, the binding to x in the comprehension bleeds outside of the
comprehension scope
Got it.
Still, this code:
...
x = 10
      [x for x in range(3)]
      print(x + 1)
will run differently in Python 2 than Python3, so even if that was a
conscious choice by the author, a --py3k flag should cause a message
for this code. This is even worse than the simpler
...
[x for x in range(3)]
      print(x + 1)
because at least that will raise a NameError when run in Python 3. The
three-line construct will still run, though produce different output.
Skip
_______________________________________________
code-quality mailing list
code-quality@python.org
https://mail.python.org/mailman/listinfo/code-quality

code-quality mailing list
code-quality@python.org
https://mail.python.org/mailman/listinfo/code-quality

PyLint catching comprehension binding leaks (was: Warn on list comprehension variables used outside list comprehensions?)

Ian Cordasco

Claudiu Popa

tags

participants (2)