Should a single/double quote followed by a left parenthesis raise SyntaxError?
I recently forgot the '%' between a format string and its tuple of values and got "TypeError: 'str' object is not callable." The error makes sense, of course, because function calling has higher precedence than anything else in the expression, so while:
'%s' 'abc'
yields '%sabc',
'%s' ('abc')
yields a TypeError. My question is whether this should be caught as a syntax error instead of a runtime error. I can't think of any possible case where a single/double quote followed by '(' (optionally separated by whitespace) would not raise an exception when the '(' is not in a string or comment. I even wrote a script (below) to check every Python file on my computer for occurrences of the regular expression r'[''"]\s*\(' occurring outside of a string or comment. The sole result of my search, a file in a wx library, is a definite runtime error by inspection: raise IndexError, "Index out of range: %d > %d" (idx, len(self._items)) Can anyone think of a counter-example? -Jake -------------------------------------------------------------------------------- #!/usr/bin/env python2 import re import sys import tokenize from py_compile import PyCompileError, compile ttypes = {v: k for k, v in tokenize.__dict__.items() if isinstance(v, int) and k == k.upper()} for filename in sys.argv[1:]: lines = file(filename, 'r').readlines() matches = [] for lnum, line in enumerate(lines, 1): for match in re.finditer(r'[''"]\s*\(', line): matches.append((lnum, match.span(), match.group(0))) try: assert(matches) compile(filename, doraise=True) except (AssertionError, PyCompileError, IOError): continue matchdict = {k: [] for k in [m[0] for m in matches]} for match in matches: matchdict[match[0]].append(match) with open(filename, 'r') as f: gen = tokenize.generate_tokens(f.readline) for ttype, tstr, (srow, scol), (erow, ecol), line in gen: if srow == erow: for mrow, (mscol, mecol), mstr in matchdict.get(srow, []): pcols = [mscol + i for i, x in enumerate(mstr) if x == '('] for p in pcols: if (p in range(scol, ecol) and ttypes[ttype] not in ['COMMENT', 'STRING']): print filename print srow print ttypes[ttype]
On 15 July 2015 at 16:40, Jacob Niehus <jacob.niehus@gmail.com> wrote:
To be honest, I'm not sure I see why this would be an improvement, given that syntax errors ("SyntaxError: invalid syntax") have less informative error messages than the TypeError you quote? Being caught at compile time doesn't seem like it's a huge benefit (and in any case, compile-time type checking is not something Python offers, in general...) Paul
On Wed, Jul 15, 2015 at 10:40 AM, Jacob Niehus <jacob.niehus@gmail.com> wrote:
It would only partially solve the problem (I think) you are trying to solve. Consider this syntactically valid case:
The TypeError tells you exactly what the problem is. A SyntaxError would catch the case where you try to call a string literal, but not a string object. Skip
On 07/15/2015 11:40 AM, Jacob Niehus wrote:
No, it's a runtime error. If it's changed there, it would need to be changed in the following places as well.
But I can see why you would think syntax error would work.
Making the other cases match that could break existing working code. And only change one of them introduces an inconsistency. Cheers, Ron
On 7/15/2015 11:40 AM, Jacob Niehus wrote:
The lexical (character) level is the wrong one for approaching this issue. Python, like most formal language interpreters, first groups characters into tokens. The above becomes 4 tokens: '%s', (, 'abc', and ). These tokens are parsed into an ast.Expr with an ast.Call object. The .fun attribute is an ast.Str and the .args attribute has the other string. The simplified version of the current grammar for a call is call ::= primary '(' args ')' Your are, in effect, pointing out that 'primary' seems too broad, and asking whether it could sensibly be replaced by something narrower. Lets see how we might define a new non-terminal 'possible_call(able)' by pulling apart 'primary': primary ::= atom | attributeref | subscription | slicing | call Each of these is a possible callable, though a callable slicing would be unusual. The latter 4 are singular categories and must be included in possible_call. But atom is itself a composite category: atom ::= identifier | literal | enclosure enclosure ::= parenth_form | list_display | dict_display | set_display | generator_expression | yield_atom Identifiers are the most common form of callable. Number and string literals never are. Parenth_forms include ( identifier ) and so possibly are*. Displays and G-Es never are (as noted by Ron Adam). I believe a yield_atom could be with gen.send(callable). * Parenth_forms include both tuples -- parentheses empty or include a comma -- and parenthesized expression -- everything else, which is to say, non-empty parentheses without a comma. It seems that these could be separate grammatical categories, in which case tuples would be excluded and parenth_expr included. So here is the proposal: in the call definition, change primary to possible_call ::= identifier | parenth_form | yield_atom | attributeref | subscription | slicing | call As for error messages, there is another thread suggesting that the messages for SyntexError might be vastly improved. -- Terry Jan Reedy
On 16 July 2015 at 05:18, Steven D'Aprano <steve@pearwood.info> wrote:
Note that the proposal should also include a change to "call": call ::= possible_call '(' args ')' replacing call ::= primary '(' args ')' (I was initially confused by the fact that possible_call included call as an option, until I remembered how it fitted into the larger picture). This seems to me like more complexity than is warranted, particularly as the error message quality drops dramatically (unless we improve the syntax error message at the same time). So I'm -0 on this. Paul
On Thu, Jul 16, 2015 at 7:00 PM, Paul Moore <p.f.moore@gmail.com> wrote:
I presume that was the intent :)
Calling the result of a call is the easiest way to demonstrate nested functions, closures, etc: def adder(x): def add(y): return x + y return add adder(5)(7) # == 12 Even if this is never used in real-world code, permitting it is a great way to show that a thing is a thing, no matter how you obtain it - you can use "adder(5).__name__" for attribute lookup on function return values, "lst[4][7]" to subscript a list item (very common!), "(yield 5)()" to call whatever function someone send()s you... worth keeping! (Also, I have unpleasant memories of PHP functions that return arrays - assigning the return value to a name and subscripting the name works, but subscripting the function return directly didn't work at the time. It has subsequently been fixed, but while I was doing that particular work, the restriction was there.)
There's another thread around the place for improving the error messages. But the message is unlikely to be accurate to the real problem anyway, as most people do not consciously seek to call string literals. It's like getting told "NoneType object has no attribute X" - the problem isn't that None lacks attributes, the problem is "why is my thing None instead of the object I thought it was". And hey. If anyone *really* wants to try to call a string lit, they can always write it thus: # Call me! response = ("0406 650 430")() That said, though, there is a backward-compatibility problem. Code which would have blown up at run-time will now fail at compile-time. It's still broken code, but people will need to check before running stuff in production. +1 on the proposal. Catching errors earlier is a good thing if it doesn't make things too complicated. ChrisA
On 16 July 2015 at 12:28, Chris Angelico <rosuav@gmail.com> wrote:
Yes, I understood that - I just wanted to make it explicit (because it confused me briefly).
Absolutely - my point wasn't that calling the result of a call is wrong, but that I had misunderstood possible_call as a replacement for call, and hence got in a muddle over the proposed grammar change - adding the explicit explanation that call is defined in terms of possible_call helped me, so I thought it might help others too. Maybe it didn't :-) Paul
On 7/16/2015 5:00 AM, Paul Moore wrote:
This was the intended meaning of "in the call definition, change primary to \n possible_call...". I should have added 'possible_call, where' to make 2 clauses like so "...to possible_call, where \n possible_call ::= <definition>". Sorry for the confusing sentence. -- Terry Jan Reedy
On Jul 15, 2015, at 16:03, Terry Reedy <tjreedy@udel.edu> wrote:
It would be fun to explain why [1,2]() is a syntax error but (1,2)() a runtime type error, and likewise for ""() vs ("")()... Anyway, I honestly still don't get the motivation here. What makes this kind of type error so special that it should be caught at compile time? We don't expect [1,2]['spam'] to fail to compile; what makes this different? It's a lot simpler to say you can try to call any primary, and you'll get a type error if the primary evaluates to something that's not callable.
On Thu, Jul 16, 2015 at 9:32 PM, Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
It makes good sense to subscript a literal list, though: month_abbr = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"][month] Calling a literal list cannot possibly succeed unless you've hacked something. The only reason that calling a tuple might survive until runtime is because it's a bit harder to detect, but Terry's original post included a footnote about further grammar tweaks that would make that possible. ChrisA
On Jul 15, 2015, at 16:03, Terry Reedy <tjreedy@udel.edu> wrote:
So here is the proposal: in the call definition, change primary to
On 7/16/2015 7:32 AM, Andrew Barnert via Python-ideas wrote: possible_call, where
Because parenthesis forms hide their contents from surrounding code. That is their purpose. In the examples above, where the content is a single object, there is no effect on the resulting ast.
Anyway, I honestly still don't get the motivation here.
Given that I switched from C to Python 18 years ago, I am obviously comfortable with runtime typing. However, I consider the delay in error detection to be a cost, not a feature. Unnoticed runtime errors sometimes sit for days to years. Earlier *is* better. The more-than-compensating benefit of dynamic typing is being able to write generic code and have user class instances participate in syntax on the same basic as with builtin classes.
What makes this kind of type error so special that it should be caught at compile time?
'SyntaxError' is defined by the grammar + a few extra compile-time checks. There are actually two versions of the grammar -- the one in the docs and the one actually used for the parser. If the grammars are tweaked to better define possibly_callable, then the invalid token sequences will be redefined as SyntaxErrors. The boundary between SyntaxError and runtime errors is somewhat arbitrary. On the other hand, one might argue that once the boundary is set, it should not change lest code be broken. Example of something that runs now and would not with the change. print 1 if False: ''() I am not sure that we have never moved the boundary, but it should be rare. Since the disallowed forms are literals and displays, I cannot think of sensible running code that that would be broken by this proposal. But the above might be enough to scuttle the idea. We don't expect [1,2]['spam'] to fail to compile; what makes this different? The compiler has a constant folder to optimize code. I currently does tuple subscriptions.
It must be that the folder discovers the error, but rather than raise immediately, it puts the 'rotten fish' back in the stream to raise a stink when encountered. I started Python with -Wall and entered the code itself and no SyntaxWarning was issued. Given that the error has been detected, but might sit silently for awhile, this seems like a shame. 'Constant' lists are not checked. They are too rare to bother checking for them.
It's a lot simpler to say you can try to call any primary, 'primary' is vague and to me way overly broad. When expanded to basic nonterminals, about half are never callable. Adding one production rule is a minor change. Leaving error detection aside, a benefit for readers would be to better define what is actually a possible callable. Of course, this could be done with a new sentence in the doc without changing the grammer. Maybe this should be the outcome of this issue. Another arguments against the proposal is that code checkers should also be able to detect such non-callable primaries. -- Terry Jan Reedy
On Jul 16, 2015, at 15:27, Terry Reedy <tjreedy@udel.edu> wrote:
'primary' is vague and to me way overly broad.
To a normal end-user, the things you syntactically call are basically all values (or, if you prefer, all expressions). Calling a value that's not callable is the same error as adding a thing that's not addable or indexing a thing that's not subscriptable, and that's a TypeError. The existing rules makes sense, and fit in with everything else in Python. For example, we don't consider {1}[1] a syntax error even though it's never valid; how is {1}(1) any different? Even with your suggested change, you still need to teach people the same rule, because s() where s is an int or (1)() are still going to be valid syntax and still going to be guaranteed type errors. All you're doing is catching the least common cause of this type error early, and I don't see who's going to benefit from that. If you're thinking there might be some benefit to teaching, or to self-taught novices, I don't see one; in act, I think it's slightly worse. They still have to learn the type rule to understand why x(0) raises at runtime--and that's far more common than 0(0). (And the current type error is pretty nice.) And if a novice can understand why x(0) is an error, they can understand why 0(0) is an error--and, in fact, they'll understand more easily if they're the same error than if they're different. And, the more someone looks into the current error, the more they'll understand. Why is 0(0) a TypeError? Because a call expression looks up the __call__ method on the object's type, and type(0) is int, and int doesn't have a __call__ method. Well, that makes sense--what would it mean to call an integer? But hey, does that mean I can create my own classes that act like functions? Cool! If you're just looking to catch errors earlier, a static type checker will catch 0(0) in exactly the same way that it will catch x(0) when x has been declared or inferred to be an int. Finally, the existing rule also has the very minor benefit that you can ast.parse 0(0) if you have a reason to do so (whether you're writing a DSL with an import hook, or writing a dev-helper tool of some kind, or even hacking up Python to add an int.__call__ method). I'm not sure why you'd want to do that (that's why it's a very minor benefit...), but I don't see any reason to make the language less consistent and stop you from doing that.
When expanded to basic nonterminals, about half are never callable. Adding one production rule is a minor change. Leaving error detection aside, a benefit for readers would be to better define what is actually a possible callable.
But why does a reader need to know what lexical forms are possibly callable? Again, the current rule is dead simple, and totally consistent with subscripting and so on. What's a possible callable? Any value is a possible callable, just as any value is a possible subscriptable and a possible addable and so on.
Of course, this could be done with a new sentence in the doc without changing the grammer. Maybe this should be the outcome of this issue.
What sentence are you envisioning here? Do we also need a sentence explaining that subscripting an int literal is always illegal despite not being a syntax error? (And that one, I can imagine a newcomer to Python actually writing, because 0[p] is valid in C...) Or that [0][0](0) is always illegal, or (0)(0)? If not, why do we need a sentence explaining that 0(0) is always illegal?
On Fri, Jul 17, 2015 at 11:44 AM, Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
To a normal end-user, the things you syntactically call are basically all values (or, if you prefer, all expressions). Calling a value that's not callable is the same error as adding a thing that's not addable or indexing a thing that's not subscriptable, and that's a TypeError. The existing rules makes sense, and fit in with everything else in Python. For example, we don't consider {1}[1] a syntax error even though it's never valid; how is {1}(1) any different?
There are other odd cases in the grammar, too, though. Try explaining this one:
If the grammar can catch an error, great! If it can't, it'll get dealt with at run-time. Syntax errors don't have to be reserved for situations that make it impossible to proceed at all. ChrisA
On Jul 16, 2015, at 18:54, Chris Angelico <rosuav@gmail.com> wrote:
If the grammar can catch an error, great! If it can't, it'll get dealt with at run-time.
This is the exact opposite. It's not the grammar sometimes catching an obvious type error earlier, it's the grammar catching something that's perfectly sensible and preventing us from writing it in the obvious way. It's something we're unfortunately forced to do because attribution syntax and float literal syntax are ambiguous. That's not something we'd want to emulate or expand on. If you really want to catch type errors at compile time, that's exactly what static type checkers (whether embedded in the compiler or not) are for; trying to hack up the grammar to do typing without doing typing is only going to catch a handful of very simple cases that nobody really cares about.
Syntax errors don't have to be reserved for situations that make it impossible to proceed at all.
No, but they should be reserved for syntactic errors, and the syntax should be as simple as possible. Making the rules more complicated and less consistent has a cost (not so much for the implementation, as for the person trying to understand the language and keep it in their head).
On 15 July 2015 at 16:40, Jacob Niehus <jacob.niehus@gmail.com> wrote:
To be honest, I'm not sure I see why this would be an improvement, given that syntax errors ("SyntaxError: invalid syntax") have less informative error messages than the TypeError you quote? Being caught at compile time doesn't seem like it's a huge benefit (and in any case, compile-time type checking is not something Python offers, in general...) Paul
On Wed, Jul 15, 2015 at 10:40 AM, Jacob Niehus <jacob.niehus@gmail.com> wrote:
It would only partially solve the problem (I think) you are trying to solve. Consider this syntactically valid case:
The TypeError tells you exactly what the problem is. A SyntaxError would catch the case where you try to call a string literal, but not a string object. Skip
On 07/15/2015 11:40 AM, Jacob Niehus wrote:
No, it's a runtime error. If it's changed there, it would need to be changed in the following places as well.
But I can see why you would think syntax error would work.
Making the other cases match that could break existing working code. And only change one of them introduces an inconsistency. Cheers, Ron
On 7/15/2015 11:40 AM, Jacob Niehus wrote:
The lexical (character) level is the wrong one for approaching this issue. Python, like most formal language interpreters, first groups characters into tokens. The above becomes 4 tokens: '%s', (, 'abc', and ). These tokens are parsed into an ast.Expr with an ast.Call object. The .fun attribute is an ast.Str and the .args attribute has the other string. The simplified version of the current grammar for a call is call ::= primary '(' args ')' Your are, in effect, pointing out that 'primary' seems too broad, and asking whether it could sensibly be replaced by something narrower. Lets see how we might define a new non-terminal 'possible_call(able)' by pulling apart 'primary': primary ::= atom | attributeref | subscription | slicing | call Each of these is a possible callable, though a callable slicing would be unusual. The latter 4 are singular categories and must be included in possible_call. But atom is itself a composite category: atom ::= identifier | literal | enclosure enclosure ::= parenth_form | list_display | dict_display | set_display | generator_expression | yield_atom Identifiers are the most common form of callable. Number and string literals never are. Parenth_forms include ( identifier ) and so possibly are*. Displays and G-Es never are (as noted by Ron Adam). I believe a yield_atom could be with gen.send(callable). * Parenth_forms include both tuples -- parentheses empty or include a comma -- and parenthesized expression -- everything else, which is to say, non-empty parentheses without a comma. It seems that these could be separate grammatical categories, in which case tuples would be excluded and parenth_expr included. So here is the proposal: in the call definition, change primary to possible_call ::= identifier | parenth_form | yield_atom | attributeref | subscription | slicing | call As for error messages, there is another thread suggesting that the messages for SyntexError might be vastly improved. -- Terry Jan Reedy
On 16 July 2015 at 05:18, Steven D'Aprano <steve@pearwood.info> wrote:
Note that the proposal should also include a change to "call": call ::= possible_call '(' args ')' replacing call ::= primary '(' args ')' (I was initially confused by the fact that possible_call included call as an option, until I remembered how it fitted into the larger picture). This seems to me like more complexity than is warranted, particularly as the error message quality drops dramatically (unless we improve the syntax error message at the same time). So I'm -0 on this. Paul
On Thu, Jul 16, 2015 at 7:00 PM, Paul Moore <p.f.moore@gmail.com> wrote:
I presume that was the intent :)
Calling the result of a call is the easiest way to demonstrate nested functions, closures, etc: def adder(x): def add(y): return x + y return add adder(5)(7) # == 12 Even if this is never used in real-world code, permitting it is a great way to show that a thing is a thing, no matter how you obtain it - you can use "adder(5).__name__" for attribute lookup on function return values, "lst[4][7]" to subscript a list item (very common!), "(yield 5)()" to call whatever function someone send()s you... worth keeping! (Also, I have unpleasant memories of PHP functions that return arrays - assigning the return value to a name and subscripting the name works, but subscripting the function return directly didn't work at the time. It has subsequently been fixed, but while I was doing that particular work, the restriction was there.)
There's another thread around the place for improving the error messages. But the message is unlikely to be accurate to the real problem anyway, as most people do not consciously seek to call string literals. It's like getting told "NoneType object has no attribute X" - the problem isn't that None lacks attributes, the problem is "why is my thing None instead of the object I thought it was". And hey. If anyone *really* wants to try to call a string lit, they can always write it thus: # Call me! response = ("0406 650 430")() That said, though, there is a backward-compatibility problem. Code which would have blown up at run-time will now fail at compile-time. It's still broken code, but people will need to check before running stuff in production. +1 on the proposal. Catching errors earlier is a good thing if it doesn't make things too complicated. ChrisA
On 16 July 2015 at 12:28, Chris Angelico <rosuav@gmail.com> wrote:
Yes, I understood that - I just wanted to make it explicit (because it confused me briefly).
Absolutely - my point wasn't that calling the result of a call is wrong, but that I had misunderstood possible_call as a replacement for call, and hence got in a muddle over the proposed grammar change - adding the explicit explanation that call is defined in terms of possible_call helped me, so I thought it might help others too. Maybe it didn't :-) Paul
On 7/16/2015 5:00 AM, Paul Moore wrote:
This was the intended meaning of "in the call definition, change primary to \n possible_call...". I should have added 'possible_call, where' to make 2 clauses like so "...to possible_call, where \n possible_call ::= <definition>". Sorry for the confusing sentence. -- Terry Jan Reedy
On Jul 15, 2015, at 16:03, Terry Reedy <tjreedy@udel.edu> wrote:
It would be fun to explain why [1,2]() is a syntax error but (1,2)() a runtime type error, and likewise for ""() vs ("")()... Anyway, I honestly still don't get the motivation here. What makes this kind of type error so special that it should be caught at compile time? We don't expect [1,2]['spam'] to fail to compile; what makes this different? It's a lot simpler to say you can try to call any primary, and you'll get a type error if the primary evaluates to something that's not callable.
On Thu, Jul 16, 2015 at 9:32 PM, Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
It makes good sense to subscript a literal list, though: month_abbr = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"][month] Calling a literal list cannot possibly succeed unless you've hacked something. The only reason that calling a tuple might survive until runtime is because it's a bit harder to detect, but Terry's original post included a footnote about further grammar tweaks that would make that possible. ChrisA
On Jul 15, 2015, at 16:03, Terry Reedy <tjreedy@udel.edu> wrote:
So here is the proposal: in the call definition, change primary to
On 7/16/2015 7:32 AM, Andrew Barnert via Python-ideas wrote: possible_call, where
Because parenthesis forms hide their contents from surrounding code. That is their purpose. In the examples above, where the content is a single object, there is no effect on the resulting ast.
Anyway, I honestly still don't get the motivation here.
Given that I switched from C to Python 18 years ago, I am obviously comfortable with runtime typing. However, I consider the delay in error detection to be a cost, not a feature. Unnoticed runtime errors sometimes sit for days to years. Earlier *is* better. The more-than-compensating benefit of dynamic typing is being able to write generic code and have user class instances participate in syntax on the same basic as with builtin classes.
What makes this kind of type error so special that it should be caught at compile time?
'SyntaxError' is defined by the grammar + a few extra compile-time checks. There are actually two versions of the grammar -- the one in the docs and the one actually used for the parser. If the grammars are tweaked to better define possibly_callable, then the invalid token sequences will be redefined as SyntaxErrors. The boundary between SyntaxError and runtime errors is somewhat arbitrary. On the other hand, one might argue that once the boundary is set, it should not change lest code be broken. Example of something that runs now and would not with the change. print 1 if False: ''() I am not sure that we have never moved the boundary, but it should be rare. Since the disallowed forms are literals and displays, I cannot think of sensible running code that that would be broken by this proposal. But the above might be enough to scuttle the idea. We don't expect [1,2]['spam'] to fail to compile; what makes this different? The compiler has a constant folder to optimize code. I currently does tuple subscriptions.
It must be that the folder discovers the error, but rather than raise immediately, it puts the 'rotten fish' back in the stream to raise a stink when encountered. I started Python with -Wall and entered the code itself and no SyntaxWarning was issued. Given that the error has been detected, but might sit silently for awhile, this seems like a shame. 'Constant' lists are not checked. They are too rare to bother checking for them.
It's a lot simpler to say you can try to call any primary, 'primary' is vague and to me way overly broad. When expanded to basic nonterminals, about half are never callable. Adding one production rule is a minor change. Leaving error detection aside, a benefit for readers would be to better define what is actually a possible callable. Of course, this could be done with a new sentence in the doc without changing the grammer. Maybe this should be the outcome of this issue. Another arguments against the proposal is that code checkers should also be able to detect such non-callable primaries. -- Terry Jan Reedy
On Jul 16, 2015, at 15:27, Terry Reedy <tjreedy@udel.edu> wrote:
'primary' is vague and to me way overly broad.
To a normal end-user, the things you syntactically call are basically all values (or, if you prefer, all expressions). Calling a value that's not callable is the same error as adding a thing that's not addable or indexing a thing that's not subscriptable, and that's a TypeError. The existing rules makes sense, and fit in with everything else in Python. For example, we don't consider {1}[1] a syntax error even though it's never valid; how is {1}(1) any different? Even with your suggested change, you still need to teach people the same rule, because s() where s is an int or (1)() are still going to be valid syntax and still going to be guaranteed type errors. All you're doing is catching the least common cause of this type error early, and I don't see who's going to benefit from that. If you're thinking there might be some benefit to teaching, or to self-taught novices, I don't see one; in act, I think it's slightly worse. They still have to learn the type rule to understand why x(0) raises at runtime--and that's far more common than 0(0). (And the current type error is pretty nice.) And if a novice can understand why x(0) is an error, they can understand why 0(0) is an error--and, in fact, they'll understand more easily if they're the same error than if they're different. And, the more someone looks into the current error, the more they'll understand. Why is 0(0) a TypeError? Because a call expression looks up the __call__ method on the object's type, and type(0) is int, and int doesn't have a __call__ method. Well, that makes sense--what would it mean to call an integer? But hey, does that mean I can create my own classes that act like functions? Cool! If you're just looking to catch errors earlier, a static type checker will catch 0(0) in exactly the same way that it will catch x(0) when x has been declared or inferred to be an int. Finally, the existing rule also has the very minor benefit that you can ast.parse 0(0) if you have a reason to do so (whether you're writing a DSL with an import hook, or writing a dev-helper tool of some kind, or even hacking up Python to add an int.__call__ method). I'm not sure why you'd want to do that (that's why it's a very minor benefit...), but I don't see any reason to make the language less consistent and stop you from doing that.
When expanded to basic nonterminals, about half are never callable. Adding one production rule is a minor change. Leaving error detection aside, a benefit for readers would be to better define what is actually a possible callable.
But why does a reader need to know what lexical forms are possibly callable? Again, the current rule is dead simple, and totally consistent with subscripting and so on. What's a possible callable? Any value is a possible callable, just as any value is a possible subscriptable and a possible addable and so on.
Of course, this could be done with a new sentence in the doc without changing the grammer. Maybe this should be the outcome of this issue.
What sentence are you envisioning here? Do we also need a sentence explaining that subscripting an int literal is always illegal despite not being a syntax error? (And that one, I can imagine a newcomer to Python actually writing, because 0[p] is valid in C...) Or that [0][0](0) is always illegal, or (0)(0)? If not, why do we need a sentence explaining that 0(0) is always illegal?
On Fri, Jul 17, 2015 at 11:44 AM, Andrew Barnert via Python-ideas <python-ideas@python.org> wrote:
To a normal end-user, the things you syntactically call are basically all values (or, if you prefer, all expressions). Calling a value that's not callable is the same error as adding a thing that's not addable or indexing a thing that's not subscriptable, and that's a TypeError. The existing rules makes sense, and fit in with everything else in Python. For example, we don't consider {1}[1] a syntax error even though it's never valid; how is {1}(1) any different?
There are other odd cases in the grammar, too, though. Try explaining this one:
If the grammar can catch an error, great! If it can't, it'll get dealt with at run-time. Syntax errors don't have to be reserved for situations that make it impossible to proceed at all. ChrisA
On Jul 16, 2015, at 18:54, Chris Angelico <rosuav@gmail.com> wrote:
If the grammar can catch an error, great! If it can't, it'll get dealt with at run-time.
This is the exact opposite. It's not the grammar sometimes catching an obvious type error earlier, it's the grammar catching something that's perfectly sensible and preventing us from writing it in the obvious way. It's something we're unfortunately forced to do because attribution syntax and float literal syntax are ambiguous. That's not something we'd want to emulate or expand on. If you really want to catch type errors at compile time, that's exactly what static type checkers (whether embedded in the compiler or not) are for; trying to hack up the grammar to do typing without doing typing is only going to catch a handful of very simple cases that nobody really cares about.
Syntax errors don't have to be reserved for situations that make it impossible to proceed at all.
No, but they should be reserved for syntactic errors, and the syntax should be as simple as possible. Making the rules more complicated and less consistent has a cost (not so much for the implementation, as for the person trying to understand the language and keep it in their head).
participants (8)
-
Andrew Barnert -
Chris Angelico -
Jacob Niehus -
Paul Moore -
Ron Adam -
Skip Montanaro -
Steven D'Aprano -
Terry Reedy