[Python-ideas] In call grammar, replace primary with possible_call? (was Re: ...quote followed by a left parenthesis...?)

[Terry Reedy]

On 7/16/2015 7:32 AM, Andrew Barnert via Python-ideas wrote:
> On Jul 15, 2015, at 16:03, Terry Reedy <tjreedy at udel.edu> wrote:
>>
>> So here is the proposal: in the call definition, change primary to
possible_call, where
>>   possible_call ::= identifier | parenth_form | yield_atom
>>                     | attributeref | subscription | slicing | call
>
> It would be fun to explain why [1,2]() is a syntax error but (1,2)() a runtime type error, and likewise for ""() vs ("")()...

Because parenthesis forms hide their contents from surrounding code. 
That is their purpose.  In the examples above, where the content is a 
single object, there is no effect on the resulting ast.

> Anyway, I honestly still don't get the motivation here.

Given that I switched from C to Python 18 years ago, I am obviously 
comfortable with runtime typing.  However, I consider the delay in error 
detection to be a cost, not a feature.  Unnoticed runtime errors 
sometimes sit for days to years. Earlier *is* better. The 
more-than-compensating benefit of dynamic typing is being able to write 
generic code and have user class instances participate in syntax on the 
same basic as with builtin classes.

 > What makes this kind of type error so special that it should be
 > caught at compile time?

'SyntaxError' is defined by the grammar + a few extra compile-time 
checks.  There are actually two versions of the grammar -- the one in 
the docs and the one actually used for the parser. If the grammars are 
tweaked to better define possibly_callable, then the invalid token 
sequences will be redefined as SyntaxErrors.

The boundary between SyntaxError and runtime errors is somewhat 
arbitrary. On the other hand, one might argue that once the boundary is 
set, it should not change lest code be broken.  Example of something 
that runs now and would not with the change.

print 1
if False: ''()

I am not sure that we have never moved the boundary, but it should be 
rare.  Since the disallowed forms are literals and displays, I cannot 
think of sensible running code that that would be broken by this 
proposal.  But the above might be enough to scuttle the idea.

We don't expect [1,2]['spam'] to fail to compile; what makes this different?

The compiler has a constant folder to optimize code.  I currently does 
tuple subscriptions.

 >>> from dis import dis
 >>> dis('(1,2)[0]')
   1           0 LOAD_CONST               4 (1)
               3 RETURN_VALUE
 >>> dis('(1,2)[1]')
   1           0 LOAD_CONST               3 (2)
               3 RETURN_VALUE
 >>> dis('(1,2)[2]')
   1           0 LOAD_CONST               2 ((1, 2))
               3 LOAD_CONST               1 (2)
               6 BINARY_SUBSCR
               7 RETURN_VALUE

It must be that the folder discovers the error, but rather than raise 
immediately, it puts the 'rotten fish' back in the stream to raise a 
stink when encountered. I started Python with -Wall and entered the code 
itself and no SyntaxWarning was issued.  Given that the error has been 
detected, but might sit silently for awhile, this seems like a shame.

'Constant' lists are not checked. They are too rare to bother checking 
for them.
 >>> dis('[1,2][0]')
   1           0 LOAD_CONST               0 (1)
               3 LOAD_CONST               1 (2)
               6 BUILD_LIST               2
               9 LOAD_CONST               2 (0)
              12 BINARY_SUBSCR
              13 RETURN_VALUE

It's a lot simpler to say you can try to call any primary,

'primary' is vague and to me way overly broad. When expanded to basic 
nonterminals, about half are never callable. Adding one production rule 
is a minor change. Leaving error detection aside, a benefit for readers 
would be to better define what is actually a possible callable.  Of 
course, this could be done with a new sentence in the doc without 
changing the grammer.  Maybe this should be the outcome of this issue.

Another arguments against the proposal is that code checkers should also 
be able to detect such non-callable primaries.

-- 
Terry Jan Reedy