Spurious unused variable messages
Code like this: import os import tempfile def func(): (fd, tmpfile) = tempfile.mkstemp(dir="/tmp") os.close(fd) print "{tmpfile}".format(**locals()) generates an unused variable warning for tmpfile, even though it's referenced in the string in the print statement. The string module has a Formatter class (since 2.6, apparently) which knows how to tear such format strings apart: >>> for elt in fmt.parse("{} {tmpfile} {1} {0:.3f}"): ... print elt ... ('', '', '', None) (' ', 'tmpfile', '', None) (' ', '1', '', None) (' ', '0', '.3f', None) I'm only now getting comfortable with the new string formatting stuff, but it seems to me that the most common use case will be to call the format method of a string literal (so this sort of usage should be fairly easy to detect). I don't think it should be terribly difficult to intersect the names coming out of Formatter.parse with the otherwise unused local variables, but I am completely unfamiliar with pylint's node traversal handlers (all the visit_* methods in variables.py). Can someone point me to some documentation for how this works, and what visit_* methods I can define? Thanks, Skip Montanaro
On Wed, Jun 10, 2015 at 2:26 PM, Skip Montanaro <skip.montanaro@gmail.com> wrote:
Code like this:
import os import tempfile
def func(): (fd, tmpfile) = tempfile.mkstemp(dir="/tmp") os.close(fd) print "{tmpfile}".format(**locals())
generates an unused variable warning for tmpfile, even though it's referenced in the string in the print statement. The string module has a Formatter class (since 2.6, apparently) which knows how to tear such format strings apart:
>>> for elt in fmt.parse("{} {tmpfile} {1} {0:.3f}"): ... print elt ... ('', '', '', None) (' ', 'tmpfile', '', None) (' ', '1', '', None) (' ', '0', '.3f', None)
I'm only now getting comfortable with the new string formatting stuff, but it seems to me that the most common use case will be to call the format method of a string literal (so this sort of usage should be fairly easy to detect). I don't think it should be terribly difficult to intersect the names coming out of Formatter.parse with the otherwise unused local variables, but I am completely unfamiliar with pylint's node traversal handlers (all the visit_* methods in variables.py). Can someone point me to some documentation for how this works, and what visit_* methods I can define?
Thanks,
Skip Montanaro
So, I'm not sure how much pylint will read into that statement. It has to recognize a few things: 1. String formatting (admittedly not that hard when the string wasn't built up, as in this example) 2. Passing and splatting locals in a call to format 3. Use of keyword arguments in the format Keep in mind, I'm not saying this shouldn't be fixed (or that it can't) just that there's a lot in play in this example beyond simply parsing the new string format. Apropos of nothing, the empty {} only works on 2.7 and beyond. 2.6 didn't support that. If you're going for 2.6 compat, you'll want to avoid that.
On Wed, Jun 10, 2015 at 3:48 PM, Ian Cordasco <graffatcolmingov@gmail.com> wrote:
So, I'm not sure how much pylint will read into that statement. It has to recognize a few things:
1. String formatting (admittedly not that hard when the string wasn't built up, as in this example)
As I stated, I think that string literal formatting will be the most common form. The token stream around that should look something like STRING-LITERAL DOT "format" (however that's spelled). What I'm initially interested in knowing is what visit_* method(s) would be involved in processing such a construct. I really don't have any idea where to begin. I think it's going to involve astroid, but I haven't found any documentation so far. In particular, I'm really only interested in the case where a variable is otherwise unused within the function, as tmpfile was in my example. I've encountered this same issue before with printf-style dict-based formatting, but I started thinking about it again when I discovered that Python's standard string module has a Formatter class which can already parse formatting strings.
2. Passing and splatting locals in a call to format
Not a big deal. In fact, I think you can completely ignore the argument to format(), certainly to start with and for this particular task.
3. Use of keyword arguments in the format
Not sure what you're getting at here.
Keep in mind, I'm not saying this shouldn't be fixed (or that it can't) just that there's a lot in play in this example beyond simply parsing the new string format.
Is it possible to build an add-on for pylint which I could mess around with? If so, and if I can learn how the basic token traversal works, then I think I could mess around without tickling core pylint.
Apropos of nothing, the empty {} only works on 2.7 and beyond. 2.6 didn't support that. If you're going for 2.6 compat, you'll want to avoid that.
Not an issue. I only mentioned 2.6 because that's when the string.Formatter class appeared. In my own environment, 2.7 is the standard.
Hi, On Thu, Jun 11, 2015 at 12:02 AM, Skip Montanaro <skip.montanaro@gmail.com> wrote:
As I stated, I think that string literal formatting will be the most common form. The token stream around that should look something like
STRING-LITERAL DOT "format"
(however that's spelled). What I'm initially interested in knowing is what visit_* method(s) would be involved in processing such a construct. I really don't have any idea where to begin. I think it's going to involve astroid, but I haven't found any documentation so far.
Unfortunately, we don't have a very good documentation, so most of the time, you can find more about something by reading the code or asking on IRC, rather than trying to find it in the documentation. That being said, regarding your use case, I don't think you actually need to implement any visit traversal function. Here's a suggestion: In checkers.variables.leave_function, when determined that there are unused variables, call a function which gathers all the string formatting calls from the function's scope, by doing something like this: for callfunc in node.nodes_of_class(astroid.CallFunc): # Infer it func = utils.safe_infer(callfunc.func) # Check if it is a string if (isinstance(func, astroid.BoundMethod) and isinstance(func.bound, astroid.Instance) and func.bound.name in ('str', 'unicode', 'bytes')): if func.name == 'format': # start parsing the string and gather the used variables. Having the list of used variables from the parsed strings, pop them out of the container of unused variables (not_consumed) and that should be it. Of course, you should check that the string formatting uses **locals.
Is it possible to build an add-on for pylint which I could mess around with? If so, and if I can learn how the basic token traversal works, then I think I could mess around without tickling core pylint.
Well, you could start playing with astroid first. Start by building an AST module using astroid.AstroidBuilder: from astroid.builder import AstroidBuilder ast = AstroidBuilder().string_build(code_string) The traversal just calls visit_<node_name> and leave_<node_name> (see pylint.utils.PyLintASTWalker).
Apropos of nothing, the empty {} only works on 2.7 and beyond. 2.6 didn't support that. If you're going for 2.6 compat, you'll want to avoid that.
Not an issue. I only mentioned 2.6 because that's when the string.Formatter class appeared. In my own environment, 2.7 is the standard.
Pylint 1.4+ doesn't support Python 2.6 anymore. Hope this info helps. /Claudiu
participants (3)
-
Claudiu Popa -
Ian Cordasco -
Skip Montanaro