[Python-Dev] Question about PEP 484

[Guido van Rossum]

Thanks, Rebecca! So the status of PEP 484 is that we don't want to add new
features to it -- those require their own PEP. But we do accept PRs for
clarifications, assuming the clarifications are about issues where the
intention is clear but the text is not (usually because we thought there
was only one possible interpretation). If PEP 484 had intended to only
allow a specific subset of Python *syntax* we surely would have specified
that subset -- so I think the intention is clear (enough).

Unfortunately a lot of PEP 484 is written in the form of examples, and in
many cases it would be possible to quibble about edge cases not covered by
the examples. (An example that just came up: is `#type: foo` a valid type
comment? What about `#   type:foo`? Or `# < type: foo`? The intention was
`#\s*type:\s*` but all the PEP examples use exactly one space.)


On Mon, Jul 16, 2018 at 4:28 PM, Rebecca Chen <rechen at google.com> wrote:

> Hi Guido and all,
>
> Let me try to clarify our (pytype team's) intentions a bit. Our
> overall goal is to see PEP 484 finalized, in the interest of having a
> definitive type-checking reference to point to and work off of. We're
> willing and eager to help with this, if we get some guidance on what
> the authors are still trying to finalize and how we can be useful. As
> a conversation starter, we offered two issues in which pytype has some
> personal stake.
>
> Best,
> Rebecca
>
> P.S. Yes, Adam manages Python at Google.
>
> On Mon, Jul 16, 2018 at 3:15 PM Guido van Rossum <guido at python.org> wrote:
> >
> > On Mon, Jul 16, 2018 at 2:56 PM, Gregory P. Smith <greg at krypto.org>
> wrote:
> >>
> >>
> >> On Mon, Jul 16, 2018 at 1:44 PM Guido van Rossum <guido at python.org>
> wrote:
> >>>
> >>> As one of the authors of PEP 484, *I* never thought there was an
> ambiguity here. The intention was for stub files to conform to the same
> grammar as regular .py files, but with a different interpretation.
> >>>
> >>> > "Have the same syntax as regular Python modules" and "are files
> containing type hints" are at odds with each other.
> >>>
> >>> That depends. *same syntax as regular Python* is normative while
> *containing type hints* is an informal description of intent.
> >>>
> >>> I happen to be somewhat familiar with the situation that lead to this
> question -- pytype has its own parser for stub files that cannot parse all
> Python constructs. But claiming that PEP 484 is ambiguous feels wrong, and
> if we really need to clarify it the only way to go is to make "same syntax
> as regular Python" more clearly normative. Type checkers should of course
> feel free ignore everything they don't care about.
> >>
> >>
> >> I feel like the "same syntax as regular Python" is too broad a
> statement.  That effectively requires a version specific Python interpreter
> to execute the files.  With at least four different Python static analyzers
> in existence today, keeping the behavior of all of them consistent is
> important.  Otherwise pyi files will be (are being) created that are
> analyzer specific and break other type checkers when distributed.
> >
> >
> > it doesn't require an interpreter, just a parser. Is saying it should be
> syntactically valid Python 3.6 (though emphatically not executable!) still
> too much?
> >
> >>
> >> ex: We're encountering pyi files with conditional logic in them.  I
> believe we've encountered pyi files with del statements in them?  Both of
> these are a slippery slope towards being turing complete in something that
> isn't supposed to be code.  I don't like this.  Interface declarations
> should not contain logic.  If we allow conditions, we need to explicitly
> define what we do allow in the PEP.  (if+else and del?  what inputs are
> allowed for the expression in if statements?).  Otherwise at some point
> someone is going to create a pyi file containing loops, function calls, and
> generator expressions and expect it to _do_ something.  The horror!  Lets
> avoid that.
> >
> >
> > Syntactically, conditional logic *is* part of the spec because of
> `sys.version_info` and `sys.platform` checks. These are mentioned in PEP
> 484 (though IIRC they were added at some point).
> >
> > Feel free to ignore `del` statements, but they are valid syntax.
> >
> >>
> >> PEP-484 does contain the text, "This also reinforces the notion that no
> runtime behavior should be expected of stub files." But reinforcing a
> notion is not what I read as a concrete statement.
> >>
> >> I'd rather see that say something like, "There must not be any runtime
> behavior from a stub file. They will be parsed for information, not
> executed."  Wordsmith that all you want, I'm not pedantic enough. :)
> >
> >
> > I don't see that as any more precise as what the PEP says. :-)
> >
> > I'd be happy to claim that if the ast module can't parse a .pyi file,
> it's invalid, otherwise it's valid, except that `# type:` comments are
> significant and the ast module doesn't preserve those.
> >
> >>
> >> I expect someone pedantic to happily point out that a def or class or
> assignment to ... with an annotation is runtime behavior... technically
> correct, but that isn't how people like me think of them in this context.
> >
> >
> > And mypy agrees. :-)
> >
> >>
> >> We use a Pythonic syntax for stubs to be consistent with the language,
> that doesn't mean they are code.
> >
> >
> > Define "code". To some people HTML is code.
> >
> >>
> >> I wrote more than I thought I would here, I'll stop now. :)
> >
> >
> > It would be nice if the pytype team could articulate the problems
> they're trying to solve, rather than offering to "help finalize PEP 484".
> My guess is that their parser for .pyi files only accepts a subset of
> Python and they're (you're? do you report to Adam?) reluctant to sink more
> time in that parser -- but I don't think that's my problem. :-)
> >
> > My intention for stubs was exactly what I've been saying in this thread:
> they must be syntactically valid Python, they are not meant to be executed,
> and type checkers are free to ignore things they don't need. If that's not
> sufficiently clear in the way the PEP is currently written, I welcome a PR
> to the peps repo. If the pytype team wants a different interpretation, the
> bar is much higher.
> >
> > --
> > --Guido van Rossum (python.org/~guido)
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180716/2cabb3c9/attachment.html>