[Typing-sig] Re: PEP 612: Parameter Specification Variables

Well, the new proposal introduces two new magic operators, versus the old proposal one new category of type variable. I don't think we really have anything similar to those magic type operators, so that seems a bigger change. (Unless you count ClassVar and Final, but those are more special -- they're not just functions from types to types.)

But frankly I'd like to see some feedback from Jukka, Ivan or Michael Sullivan.

On Wed, Feb 5, 2020 at 7:48 PM Mark Mendoza mendoza.mark.a@gmail.com wrote:

Hi Guido, Thanks for volunteering as a sponsor!

I think there is one more issue I'd like to get feedback on: a more thorough discussion of Ran Benita's ParametersOf proposal. After some internal discussion, we realized that the more apt analogy to TypedScript would be of an operator that was defined on other types, not names. If that type is a type variable, you can express everything you can in ParamSpecs in terms of ParametersOf operations.

The trick is defining what the bound on the type variable has to be. So far we have been avoiding “late incompatibility”, or errors that occur during type variable instantiation. This means that we need to somehow be able to verify that ParametersOf[X] will actually be able to extract a parameter specification from any X that would be substituted into it. To achieve this, we need a bound for the type variable that includes exactly all callables. Currently there are no types in the type system that fit that bill while not introducing an “Any”. The closest thing we have is Callable[..., object], but our current semantic for that is that we permit it to be called with any arguments, unsoundly.

Therefore this proposal requires the definition of a new type, Function. Function is callable with, and only with ParametersOf[F]. ParametersOf can only operate on type variables with precisely this bound. This means we’re just introducing a new bound and a new operator (one more if you also want ReturnType[F], but that’s actually not strictly necessary, see case #3 below), rather than a new kind of type variable.

To demonstrate the pros and cons of this alternative, here are some examples of translating between the two forms:

In all of the following examples:

TParams = ParamSpec("TParams")
TReturn = TypeVar("TReturn")
F = TypeVar("F", bound=Function)

#1 Typing a type-preserving decorator

def no_change_decorator(
   f: Callable[TParams, TReturn]
) -> Callable[TParams, TReturn]:
    def inner(*args: TParams.args, **kwargs: TParams.kwargs) -> TReturn:
       return f(*args, **kwargs)
    return inner

vs.

def no_change_decorator(f: F) -> F:
    def inner(
      *args: ParametersOf[F].args,
      **kwargs: ParametersOf[F].kwargs
    ) -> ReturnType[F]:
       return f(*args, **kwargs)
    return inner

#2 Typing a decorator that wraps the return type in a List

def wrapping_decorator(
    f: Callable[TParams, TReturn]
) -> Callable[TParams, List[TReturn]]:
    def inner(
      *args: TParams.args,
      **kwargs: TParams.kwargs
    ) -> List[TReturn]:
       return [f(*args, **kwargs)]
    return inner

vs.

def wrapping_decorator(
    f: F
) -> Callable[ParametersOf[F], List[ReturnType[F]]]:
    def inner(
        *args: ParametersOf[F].args,
        **kwargs: ParametersOf[F].kwargs
     ) -> List[ReturnType[F]]:
       return [f(*args, **kwargs)]
    return inner

#3 Typing a decorator that unwraps a list return type

def unwrapping_decorator(
    f: Callable[TParams, List[TReturn]]
) -> Callable[TParams, TReturn]:
    def inner(*args: TParams.args, **kwargs: TParams.kwargs) -> TReturn:
       return f(*args, **kwargs)[0]
    return inner

vs.

def unwrapping_decorator(
    f: Callable[ParametersOf[F], List[TReturn]]
) -> Callable[ParametersOf[F], TReturn]:
    def inner(
       *args: ParametersOf[F].args,
       **kwargs: ParametersOf[F].kwargs
    ) -> TReturn:
       return f(*args, **kwargs)[0]
    return inner

#4 Typing a class that is a container for a callable

class FunctionContainer(Generic[TParams, TReturn]):
    f : F
    def __init__(self, f: Callable[TParams, TReturn]) -> None:
        self.f = f
    def call(
       self,
       *args: TParams.args,
       **kwargs: TParams.kwargs
    ) -> TReturn:
        return self.f(*args, **kwargs)

vs.

class FunctionContainer(Generic[F]):
    f : F
    def __init__(self, f: F) -> None:
        self.f = f
    def call(
        self,
        *args: ParametersOf[F].args,
        **kwargs: ParametersOf[F].kwargs
    ) -> ReturnType[F]:
        return self.f(*args, **kwargs)

In my opinion, case #1 is the best one for the proposed alternative, and case #3 is the worst one. Case #3 is odd to read because F is not actually referring to any callable. It’s just being used as a container for the ParameterSpecification. I think that the weakness in case #3 highlights what exactly the difference in philosophy between these two approaches is. ParamSpec was born out of the existing approach to Python typing which so far has avoided supporting operators, whether user-defined or built-in, in favor of destructuring. ParametersOf was inspired by TypeScript, which has a type system with extensive use of user-defined operators. Case #3 would be much more ergonomic if you could define an operator RemoveList[List[X]] = X and then you could just return Callable[ParametersOf[F], RemoveList[ReturnType[F]]]. Without that, you unfortunately get into a situation where you have to use a F-variable like a ParamSpec, in that you never actually bind the return type. Overall, the oddness of Case #3 in the ParametersOf alternative is the biggest issue with the ParametersOf proposal that we've heard internally. Combining destructuring with operators does seem to genuinely cause some ergonomics/readability issues.

Another issue we've heard raised from Python users we've shown this to is that it would potentially be confusing to have multiple ways to spell the same equivalent type i.e. F === Callable[ParametersOf[F], ReturnType[F]].

The big win for the alternative, beyond the concision of case #1, is that it would avoid introducing a new kind of type variable could potentially make further extensions to the language more easy to specify.

I can see the merits of both syntaxes, so I'd love to get some feedback from the community about folks' preferences before we proceed with the approval process.

Thanks again for your support and sponsorship!

Best, Mark Mendoza _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/