[Python-ideas] In call grammar, replace primary with possible_call? (was Re: ...quote followed by a left parenthesis...?)

[Terry Reedy]

On 7/15/2015 11:40 AM, Jacob Niehus wrote:
> I recently forgot the '%' between a format string and its tuple of
> values and got "TypeError: 'str' object is not callable." The error
> makes sense, of course, because function calling has higher precedence
> than anything else in the expression, so while:
>
>>>> '%s' 'abc'
>
> yields '%sabc',
>
>>>> '%s' ('abc')
>
> yields a TypeError.
>
> My question is whether this should be caught as a syntax error instead
> of a runtime error. I can't think of any possible case where a
> single/double quote followed by '(' (optionally separated by whitespace)
> would not raise an exception when the '(' is not in a string or comment.

The lexical (character) level is the wrong one for approaching this 
issue.  Python, like most formal language interpreters, first groups 
characters into tokens.  The above becomes 4 tokens: '%s', (, 'abc', and 
).  These tokens are parsed into an ast.Expr with an ast.Call object. 
The .fun attribute is an ast.Str and the .args attribute has the other 
string.

The simplified version of the current grammar for a call is
   call  ::= primary '(' args ')'
Your are, in effect, pointing out that 'primary' seems too broad, and 
asking whether it could sensibly be replaced by something narrower.

Lets see how we might define a new non-terminal 'possible_call(able)' by 
pulling apart 'primary':
   primary ::= atom | attributeref | subscription | slicing | call
Each of these is a possible callable, though a callable slicing would be 
unusual. The latter 4 are singular categories and must be included in 
possible_call.  But atom is itself a composite category:

   atom      ::=  identifier | literal | enclosure
   enclosure ::=  parenth_form | list_display | dict_display
                  | set_display | generator_expression | yield_atom

Identifiers are the most common form of callable.  Number and string 
literals never are. Parenth_forms include ( identifier ) and so possibly 
are*. Displays and G-Es never are (as noted by Ron Adam).  I believe a 
yield_atom could be with gen.send(callable).

* Parenth_forms include both tuples -- parentheses empty or include a 
comma -- and parenthesized expression -- everything else, which is to 
say, non-empty parentheses without a comma.  It seems that these could 
be separate grammatical categories, in which case tuples would be 
excluded and parenth_expr included.

So here is the proposal: in the call definition, change primary to
   possible_call ::= identifier | parenth_form | yield_atom
                     | attributeref | subscription | slicing | call

As for error messages, there is another thread suggesting that the 
messages for SyntexError might be vastly improved.

-- 
Terry Jan Reedy