I played around more with the strategy of using an additional scope but passing in all referenced symbols as arguments ("lambda lifting"). This approach unfortunately adds _significant_ complexity to the compiler. It also doesn't address some edge cases such as the use of walrus operators in the inner scope. I'm also worried about the runtime overhead and the impact on compatibility with debuggers and runtime type checking libraries. For those reasons, I abandoned the approach.
The good news is that I think I've come up with a solution that addresses all of the requirements, doesn't add significant complexity to the compiler, and shouldn't have any compatibility impact on debuggers. The key is to abandon the requirement that every reference to a type variable "T" refer to the same object. Instead, I have the compiler generate code to construct a new "type var proxy" object each time "T" is referenced. Since type variables are typically referenced only in type annotations (which are evaluated once — or never if "from __future__ import annotations" is in effect) and in specialized base classes, I'm not concerned about the runtime overhead of allocating one of these objects each time.
This limits the compiler changes to the following: 1. The symtable.c module needs to track which type parameters are "active" as it recursively explores the AST. If it sees a local symbol name (a local variable, a parameter in a function scope, a class variable in a class scope, etc.) that conflicts with one of the active type parameters, it emits a syntax error. 2. When the compiler.c module is generating byte codes and comes across a name that was marked as a type parameter by the symtable.c module, it emits byte codes to construct a "typing.TypeParameter(name)" object. 3. When the compiler.c module emits byte codes for a class statement with new-style type parameters, it implicitly adds "Generic" to the base class list.
I have this all working in a fork of cpython if you want to check it out: https://github.com/erictraut/cpython/tree/type_param_syntax
I can go over the details in tomorrow's typing meetup discussion.
I've updated the draft PEP to reflect these changes. I've also made the following additional changes based on feedback: 1. I've scaled back the proposal and removed sections on "default type arguments", explicit specialization of generic functions, and recursive type aliases. These can be covered in other PEPs. 2. I've switched from an "extends" keyword to a simple colon when specifying the upper bound or the constraints for the type variable. This feels more Pythonic.
I've implemented provisional support for most of this in pyright (currently in a private branch) to prove out the type checking side of it. I can demo this in tomorrow's typing meetup.
-- Eric Traut Contributor to Pyright & Pylance Microsoft