Mailman 3 July 2021 - Typing-sig

by Adrian Freund

Hello typing-sig, I wanted to suggest adding support for sealed classes to the python type system. What are sealed classes? Sealed classes are classes of which all subclasses are known statically. This is usually accomplished by requiring all subclasses to be created in the same file/module as the parent class. What languages support sealed classes? Sealed classes are supported by C#, Kotlin and starting with Java 15 even by Java. What can sealed classes be used for? Sealed classes are useful for checking the exhaustiveness of if/else towers and (python 3.10) match statements for example when working with abstract syntax trees. Take a look at this example: ``` class Node(ABC): ... class Value(Node): ... class BinOp(Node): ... class UnOp(Node): ... n: Node match n: case Value(v): ... case BinOp(n1, op, n2): ... case UnOp(op, n1): ... # Can we statically check if all cases are handled? ``` Sealed classes in python: My suggestion would be to add a new class "Sealed" to typing and typing_extensions that can be inherited from: class Node(Sealed): ... Type checkers should make sure that no subclasses of Node are created in modules other than the module Node was created in and should treat sealed classes as abstract classes. Additionally they can keep track of all the subclasses of Node do implement exhaustiveness checking. The runtime implementation of sealed could also prevent non-typechecked code from creating additional subclasses: ``` class Sealed(ABC): def __init_subclass__(cls): for c in cls.__mro__: if Sealed in c.__bases__ and c.__module__ != cls.__module__: raise RuntimeError('Sealed class can\'t be subclassed from another module. Sealed in module "{}".'.format(c.__module__)) ``` Final example: a.py: ``` from typing import Sealed class Base(Sealed): pass class A(Base): pass class B(A): pass ``` b.py: ``` from a import Base, A class C(Base): # raises an exception pass class D(A): # also raises an exception pass ``` Adrian Freund

1 month

5
7
0 0

Thoughts on a typing.Self?

by gobot1234yt＠gmail.com

I would like to draft a PEP for a typing.Self class which would perform similarly to rust's Self keyword in function annotations. Examples ```python from typing import TypeVar F = TypeVar("F", bound="Foo") class Foo: def parse(self: F, data: bytes) -> F: ... return self ``` vs ```python from typing import Self class Foo: def parse(self, data: bytes) -> Self: ... return self ``` Another example of where this could be useful is classmethods: ```python class Foo: @classmethod def bar(cls: type[F], *args: Any, **kwargs: Any) -> F: return cls() ``` vs ```python class Foo: @classmethod def bar(cls, *args: Any, **kwargs: Any) -> Self: return cls() ``` Implementation wise it would be a copy of NoReturn, i.e. not subscriptable, a _SpecialForm and should only be valid as a return type. I haven't seen any discussion of the idea on the mailing list, so I wanted to run it by everyone. Thanks, James.

1 month

12
26
1 0

Compact syntax for Callable: `(Arg1, Arg2) -> Ret`

by Steven Troxler

Hi typing-sig, I’d like to start a discussion about moving forward with improved syntax for typing.Callable based on discussion in the Typing Summit: * Allow `(ArgType0, ArgType1) -> ReturnType` * In place of `Callable[[ArgType0, ArgType1], ReturnType]` A lot of folks also want a way to express things like named, optional, and variadic args, but this is more complicated and has harder tradeoffs. I'll start a separate thread next week to discuss that, let's aim to keep this thread focused on the syntactic sugar described above. If we think this is a good idea, I'm happy to work on drafting a PEP. ## Background The existing Callable type is useful for simple functions with required arguments passed by position, but there’s consensus that it can be hard to keep track of brackets, especially as the types get complex. For example, even in very simple functional-style code with higher-order functions the brackets can be cumbersome: ``` from typing import Callable def accepts_callback(callback: Callable[[str, int], str]) -> None: print(callback("answer", 42) def f(s: str, x: int) -> str: return s + ": " + str(x) accepts_callback(f) accepts_callback(lambda s, _: s) ``` In agreement with a poll from the Typing Summit, I’d like to propose allowing * `(Argtype1, Argtype2) -> ReturnType` to be used in place of * `Callable[[Argtype1, Argtype2], ReturnType]` This allows us to rewrite the example above as: ``` def accepts_callback(callback: (str, int) -> str) -> None: print(callback("answer", 42) def f(s: str, x: int) -> str: return s + ": " + str(x) accepts_callback(f) accepts_callback(lambda s, _: s) ``` Advantages of arrow syntax include: * eliminate the need to import Callable * easier-to-skim types * they are shorter, and one of the problems with Callables is just verbosity * the parentheses and arrow have more visual structure than existing bracket-based Callable syntax * a syntax that should feel familiar: * it mimics the syntax for ordinary functions and stubs * it is similar to several other languages e.g. * [typescript](https://basarat.gitbook.io/typescript/type-system/callable#arrow-syntax) * [flow](https://flow.org/en/docs/types/functions/#toc-function-types) * [kotlin](https://kotlinlang.org/docs/lambdas.html#function-types) ## Open questions about syntax rules Would we require parentheses around a callable return type as above, or allow it to be bare (which causes two ->s to be visually at the same level of the code, even though one is nested in the AST)? Consider this function (annotated using a conservative syntax where we wrap the return type in paranetheses): ``` def identity( f: (str, int, int) -> int, ) -> ((str, int, int) -> int): return f ``` In this case, would we want to * require parentheses around a callable return type as above, or * allow it to be bare (which causes two ->s to look a little bit as if they are at the same code level) Here’s what the example above would look like if we allow bare arrow syntax in the return type: ``` def identity( f: (str, int, int) -> int, ) -> (str, int, int) -> int: return f ``` Another option, if we feel like the outer parentheses are necessary would be to make the parameter grouping optional in this case, for example: ``` def identity( f: (str, int, int) -> int, ) -> (str, int, int -> int): return f ``` ## Alternatives considered We ran a poll at the typing summit comparing three options for nicer Callable syntax: * This proposal `(ArgType1, ArgType2, ArgType3) -> ReturnType` * 72% of participants upvoted this * A typescript-like syntax `(a: ArgType1, b: ArgType2) => ReturnType` * 62% of participants upvoted this * Disadvantages are * a new symbol `=>` * argument names are nuisance parameters for positional args (which is all the existing Callable supports) * `[ArgType1, ArgType2, ArgType3] -> ReturnType` * 8% of participants upvoted this * The main disadvantage is that brackets don’t seem as intuitive - Steven

1 month

10
55
2 0

Support for type checking fixed width integers

by Arun Sharma

I'm aware that the python3 type system primarily supports arbitrary precision integers and there is no support for type checking fixed width integers. However, when python is used as an approachable first language for data scientists, scientists interested in numerical analysis and possibly other generic use cases, one common technique is to map python's type system to that of a lower level underlying system. I'm primarily coming at it from the third use case in the form of py2many, a transpiler for converting python to many statically typed C-like languages. I suspect pytorch, tensorflow or anything that maps python to vectorized hardware efficiently has to deal with the precision of integers. Canonical use case: ``` #!/usr/bin/env python3 from ctypes import c_int8 as i8 def test(): a: i8 = 10 b: i8 = 20 c = a + b reveal_type(c) ``` Also would love to have type checkers infer ranges and throw errors for cases like: c: i8 = 300 Doing so would significantly benefit the use case of generating secure C++/Rust code from python source: https://github.com/adsharma/py2many/issues/233 Eric Traut suggested I post to the mailing list. More context here: https://github.com/microsoft/pyright/issues/1872 -Arun

1 month, 1 week

2
4
0 0

Type bounds on PEP 646

by Alfonso L. Castaño

Hi, I have been thinking about the implications of not supporting bounds in PEP 646. Although when I mentioned it the last time that I checked the PEP I did not give it much importance, after having thought more about it and heard about more use cases I wonder if leaving the bound for another PEP would not limit too much the value of the PEP. The main motivation of the PEP is to use variadics to model the shape of a tensor, either by specifying the exact size of the dimension or by specifying their name. However, if for example a library like NumPy or PyTorch would want to take advantage of this feature, they would need to restrict the bound of the variadic. For example, if they go for typing exact dimensions they would like to accept Tensor[L[20],L[40]] but not Tensor[L[20], str, ClassFoo]. (L = Literal) Similarly, if they prefer to go for the naming route (Tensor[Width, Height]) but they don't want arbitrary types in there. Therefore, I was wondering if leaving it for a future PEP is the best approach. The initial reason was to keep the PEP simple but I think that the additional complexity could come from supporting Variance (which can be left out), but bounds would not introduce any new behavior, they would just replicate the logic of the bounds in TypeVar. Any thoughts? Alfonso.

1 month, 2 weeks

4
12
0 0

Adapting Callable type signatures at static analysis time

by John Snow

Hi, I am not sure if there's a way to do what I want, I'd love some pointers or corrections to my thinking. Let's say I have some unbound method F: F: Callable[[T, ...], U] i.e. method F belongs to some class T and takes ... some number and type of arguments. It returns an object of type U. (As far as I am aware, this is not a legitimate use of ellipsis here.) I'd like to annotate a decorator such that the return type is: F': Callable[T', ...], U] i.e. F' is a function that belongs to some class T' (where T' is not <= T ...!), but accepts the same number and type of arguments as the original method. I haven't found a way to capture the variadic nature of "all of the arguments to a function" in order to "unwrap" a callable's type at static analysis time. Is there a way to do so? I've found ways to capture the entire callable's type (as in the documentation for how to annotate decorators) but not any techniques for decomposing the callable. Forgive the clunky syntax, but I'd love to find some way to do something like this: Callable[[T, V...], U] Where "V..." stands for some particular set and type of arguments to a callable after "T" has been applied, which lets me do "surgery" on Callable types and adapt them at static analysis time such that the transformed/wrapped versions have typing information available to e.g. mypy. Is this even vaguely achievable? Maybe this is the sort of thing that PEP 646 was meant to address and I just need to sit on my hands for now. Thanks, --js

1 month, 2 weeks

2
1
0 0

Follow-up on Pyre implementation of Broadcast and Literal Arithmetic

by Alfonso L. Castaño

Thank you again Pradeep and Brandon for the presentation last Monday! It is great to see the progress of Pyre and I appreciate the effort that you make for implementing new features in Pyre. I wanted to make sure that I fully understand what was presented in case that we can provide additional feedback. Regarding the implementation in Pyre of the Broadcast operator, I was wondering if the implementation follows the proposal that I made in 14-09-2020 (https://docs.google.com/presentation/d/16VGNfVo6Kxk9lBhm1cx1-Jr19dqESyy10Wk…) or if you have made any change on its behavior or properties. For example, you mentioned that you were doing some normalization even if it was not in the slides, so I was curious if maybe you have spotted new equivalences of Broadcasting expressions. And as of literal arithmetic, I was wondering if it is available in Pyre. Although I said in (https://docs.google.com/presentation/d/16VGNfVo6Kxk9lBhm1cx1-Jr19dqESyy10Wk…) that it requires standardization to be ready (typeshed) but I was wondering if it is already supported by rewriting typeshed internally. Best, Alfonso.

2 months

2
1
0 0

Subtyping and Any

by Kevin Millikin

I'm trying to sort out the Python typing rules using PEP-483 as a starting point and the treatment of is-consistent-with and Any seems unclear. "The is-consistent-with relationship is defined by three rules: - A type t1 is consistent with a type t2 if t1 is a subtype of t2. (But not the other way around.) - Any is consistent with every type. (But Any is not a subtype of every type.) - Every type is consistent with Any. (But every type is not a subtype of Any.)" And then, for instance: "Tuple[u1, u2, ..., um] is a subtype of Tuple[t1, t2, ..., tn] if they have the same length n==m and each ui is a subtype of ti." From the above, I would expect type checkers to reject both of the assignments below: t0: Tuple[int, Any] = (0, object()) t1: Tuple[int, str] = (1, 't1') t0 = t1 t1 = t0 Neither of these types are Any so they are only consistent if they have a subtype relationship. Mypy and pytype accept both assignments. If we treat Any as a top type then the first assignment is OK and the second is not, and that's not what's going on. So there must be some notion of consistent subtyping that is what is actually implemented. Is this spelled out anywhere other than the code? -- [image: logo] Kevin Millikin Software Engineer kmillikin(a)deepmind.com If you received this email in error, please do not forward it to anyone else (it may contain confidential or privileged information), erase all copies and attachments, and let me know that it has gone to the wrong person.

2 months

3
7
0 0

Why is Type[Type] disallowed? (repost)

by Buck Evan

Now with improved legibility: PEP-0484 specifies that Any, classes, unions of classes, and type variables can be parameters of types, and "Any other special constructs like Tuple or Callable are not allowed as an argument to Type." I found the origin of this text: https://github.com/python/typing/pull/218#discussion-diff-63608361 It looks like the intent of this was to prohibit non-class values, but the improved wording was overly strict. mypy uses Type[Type[]] in its internal representation and error messages already: ``` list_str: List[str] = ["a", "b", "c"] type_list_str: Type[List[str]] = type(list_str) type_type_list_str = type(type_list_str) reveal_type(type_type_list_str) # note: Revealed type is 'Type[Type[builtins.list[builtins.str]]]' ``` and will typecheck such an annotation as expected: ``` list_str: List[str] = ["a", "b", "c"] type_list_str: Type[List[str]] = type(list_str) # error: Incompatible types in assignment (expression has type "Type[Type[List[str]]]", variable has type "Type[Type[List[int]]]") type_type_list_int: Type[Type[List[int]]] = type(type_list_str) ```

2 months, 1 week

1
0
0 0

Why is Type[Type] disallowed?

by Buck Evan

PEP-0484 specifies that Any, classes, unions of classes, and type variables can be parameters of types, and "Any other special constructs like Tuple or Callable are not allowed as an argument to Type." I found the origin of this text: https://github.com/python/typing/pull/218#discussion-diff-63608361 It looks like the intent of this was to prohibit non-class values, but the improved wording was overly strict. mypy uses Type[Type[]] in its internal representation and error messages already: ``` list_str: List[str] = ["a", "b", "c"] type_list_str: Type[List[str]] = type(list_str) type_type_list_str = type(type_list_str) reveal_type(type_type_list_str) # note: Revealed type is 'Type[Type[builtins.list[builtins.str]]]' ``` and will typecheck such an annotation as expected: ``` list_str: List[str] = ["a", "b", "c"] type_list_str: Type[List[str]] = type(list_str) # error: Incompatible types in assignment (expression has type "Type[Type[List[str]]]", variable has type "Type[Type[List[int]]]") type_type_list_int: Type[Type[List[int]]] = type(type_list_str) ```

2 months, 1 week

1
0
2 0

2021

2020

2019

2018

Typing-sig July 2021