[Python-Dev] Question about PEP 484

[Guido van Rossum]

On Mon, Jul 16, 2018 at 2:56 PM, Gregory P. Smith <greg at krypto.org> wrote:

>
> On Mon, Jul 16, 2018 at 1:44 PM Guido van Rossum <guido at python.org> wrote:
>
>> As one of the authors of PEP 484, *I* never thought there was an
>> ambiguity here. The intention was for stub files to conform to the same
>> grammar as regular .py files, but with a different interpretation.
>>
>> > "Have the same syntax as regular Python modules" and "are files
>> containing type hints" are at odds with each other.
>>
>> That depends. *same syntax as regular Python* is normative while
>> *containing type hints* is an informal description of intent.
>>
>> I happen to be somewhat familiar with the situation that lead to this
>> question -- pytype has its own parser for stub files that cannot parse all
>> Python constructs. But claiming that PEP 484 is ambiguous feels wrong, and
>> if we really need to clarify it the only way to go is to make "same syntax
>> as regular Python" more clearly normative. Type checkers should of course
>> feel free ignore everything they don't care about.
>>
>
> I feel like the "same syntax as regular Python" is too broad a statement.
> That effectively requires a version specific Python interpreter to execute
> the files.  With at least four different Python static analyzers in
> existence today, keeping the behavior of all of them consistent is
> important.  Otherwise pyi files will be (are being) created that are
> analyzer specific and break other type checkers when distributed.
>

it doesn't require an interpreter, just a parser. Is saying it should be
syntactically valid Python 3.6 (though emphatically not executable!) still
too much?


> ex: We're encountering pyi files with conditional logic in them.  I
> believe we've encountered pyi files with del statements in them?  Both of
> these are a slippery slope towards being turing complete in something that
> isn't supposed to be code.  I don't like this.  Interface declarations
> should not contain logic.  If we allow conditions, we need to explicitly
> define what we do allow in the PEP.  (if+else and del?  what inputs are
> allowed for the expression in if statements?).  Otherwise at some point
> someone is going to create a pyi file containing loops, function calls, and
> generator expressions and expect it to _do_ something.  The horror!  Lets
> avoid that.
>

Syntactically, conditional logic *is* part of the spec because of
`sys.version_info` and `sys.platform` checks. These are mentioned in PEP
484 (though IIRC they were added at some point).

Feel free to ignore `del` statements, but they are valid syntax.


> PEP-484 does contain the text, "This also reinforces the notion that no
> runtime behavior should be expected of stub files." But reinforcing a
> notion is not what I read as a concrete statement.
>
> I'd rather see that say something like, "There must not be any runtime
> behavior from a stub file. They will be parsed for information, not
> executed."  Wordsmith that all you want, I'm not pedantic enough. :)
>

I don't see that as any more precise as what the PEP says. :-)

I'd be happy to claim that if the ast module can't parse a .pyi file, it's
invalid, otherwise it's valid, except that `# type:` comments are
significant and the ast module doesn't preserve those.


> I expect someone pedantic to happily point out that a def or class or
> assignment to ... with an annotation is runtime behavior... technically
> correct, but that isn't how people like me think of them in this context.
>

And mypy agrees. :-)


> We use a Pythonic syntax for stubs to be consistent with the language,
> that doesn't mean they are code.
>

Define "code". To some people HTML is code.


> I wrote more than I thought I would here, I'll stop now. :)
>

It would be nice if the pytype team could articulate the problems they're
trying to solve, rather than offering to "help finalize PEP 484". My guess
is that their parser for .pyi files only accepts a subset of Python and
they're (you're? do you report to Adam?) reluctant to sink more time in
that parser -- but I don't think that's my problem. :-)

My intention for stubs was exactly what I've been saying in this thread:
they must be syntactically valid Python, they are not meant to be executed,
and type checkers are free to ignore things they don't need. If that's not
sufficiently clear in the way the PEP is currently written, I welcome a PR
to the peps repo. If the pytype team wants a different interpretation, the
bar is much higher.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180716/d78d48e9/attachment.html>