On Sat, 18 Dec 2021 at 12:14, Ken Jin <kenjin4096@gmail.com> wrote:
Hi Paul, to clarify, I didn't mean to insinuate that we can break backwards compatibility in typing.py, and I apologize if it seemed that way. I was replying to Steven's claim that none of typing is marked as 'Provisional'. Maybe I shouldn't have been such a pedant ;-). Like I stated above, typing.py (the module itself) is non-provisional.
No apology needed, I'm all in favour of pedantry :-) But to be serious, people (like myself) who are outsiders will not be aware of the nuances, and will see the PEP as "Provisional" and make inferences based on that. Which probably isn't what you want. We have the same issue in packaging, where we *also* use the PEP system for purposes outside of its original intent. We've been hit by *precisely* this issue, as PEP 517 (one of the packaging interface specs) was for a long time left as "Provisional", and people outside of the packaging community took that as meaning it "wasn't ready", which delayed adoption of important new features. It wasn't a disaster, but it was definitely something we should have avoided, by being more careful to mark the PEP as "Final".
A few of us tried to change PEP 484 to "Final", but we realized the PEP was too loaded. Please see our discussion here https://github.com/python/peps/pull/1942. I'll try my best to summarize (adapted/paraphrased from Jelle's response, mixed with mine, thanks for his insight!):
The comment "one reason PEP 484 is a bit special is that for a lot of things it's the only specification" from that thread is particularly interesting, as we had precisely the same issue with PEP 517 (it's *still* the only documentation of the particular interface it defines), and we had to be careful with clarifications and extensions (we basically approved the changes as a "PEP update" and included them in a "Summary of changes" section. It's not precisely in line with the standard process, but it worked well enough in this case.
PEP 484 is a bit special. Apart from specifying typing.py's behavior, it's also a specification for what all type checkers should do. There are no other documents specifying the behavior of type checkers as extensively as PEP 484, (the typing docs in CPython don't cut it). The problem here is that if we discover some wart or edge case in PEP 484's type checker behavior (and we still do every year!), we need to update it. Marking it as "Final" would mean we can no longer change PEP 484 to fix things in this manner.
Yeah, that's very similar to packaging interoperability specs. I'd strongly recommend adding at least a "Summary of Changes" section to PEP 484, so people can easily see the evolution, and so that it's more visible if the level of change gets too high (repeated small changes often look far less of an issue than the cumulative result actually is).
Some of us proposed to decouple the type checker specification from PEP 484, but that's a huge undertaking.
Yeah, big reworks of documentation like that are good in theory, but not actually practical. All of this sounds very like the sorts of things packaging has encountered. Our solutions aren't by any means perfect, and there's lots of differences which mean what we do might not suit the typing community's needs, but there's a lot of overlap, and I'm sure we could learn from each other. Paul