Hello all,
User-friendly Callable syntax is something we all want to see. However,
there has been no real movement on the discussion in the last one month.
People have had differing opinions based on design principles, stats, and
personal experience. This has led to half a dozen proposals with some
subtle and not-so-subtle differences [1]. And Guido pointed out that we
want to settle on a full proposal instead of going for a partial PEP now
and settling on the remaining details in a later PEP.
How about we meet up over a zoom call and discuss some of the key concerns?
Things proceed noticeably faster when people from the typing community are
able to get in the same (virtual) room (I saw that with PEP 646 and the
typing summit). We don't have to reach a consensus in the first meeting
itself. It's more a way to break the deadlock and make progress.
Date: 10 am San Francisco time (6 pm UK time), Monday, September 13, 2021.
Date is flexible.
Duration: 1 hour.
If there is enough interest, I can set up the video call and share the link
and agenda. If you're interested in presenting pros and cons for your
favorite proposal, reply to this mail.
[1]: Document containing the various proposals and use cases:
https://docs.google.com/document/d/11VkNk9N8whxS1SrBv2BNN1Csl5yeTlgG7WyLrCB…
--
S Pradeep Kumar
Hi,
I’m in a position, where I’d like to be able to pass a reference to a
Protocol Class, however, I can’t find a way to describe that variable with
the current type system, as Type[ProtocolClass] means a class implementing
said protocol, where I want the protocol itself, while ProtocolClass means
an instance of a class implementing the protocol.
So I haven’t figured out if PEP 544 specifies a type that describes the
Protocol class itself, rather than just its compatible implementations.
I tried to file this as a mypy bug
<https://github.com/python/mypy/issues/10988>, but no response yet, and
alas, figure it may even be overlooked by the PEP in question.
Anyway, I’d very much appreciate a few more eyes and input on this matter.
Thank you,
Cheers
This is a cross post from
https://discuss.python.org/t/how-to-pass-a-reference-to-a-protocol-class/10…
Reported MyPy Issue: https://github.com/python/mypy/issues/10988
Is anything like mypy's `NamedArg` coming to the Python standard
library? Meanwhile, if I want to include named arguments in a Callable
type, is the right approach to use something like `Callable[..., int]`?
This was previously briefly discussed in
https://github.com/python/typing/issues/601.
Occasionally it's useful if it was possible to mark protocol fields as
optional. While the existence of such a field is not guaranteed, if it
exists, it follows the described protocol. This is especially useful for
I/O protocols, where implementations often check for the existence of a
particular method, before using it. One prominent example is the WSGI
spec, where the iterable returned by an application can optionally have
a close() method.
My suggestion is to add an @optional decorator to typing to be used like
this:
from typing import Protocol, optional
class MyProto(Protocol):
def write(self, __b: bytes) -> Any: ...
@optional
def close(self) -> Any: ...
On the Python side, this is fairly trivial to implement: @optional would
just return the passed in function, possible adding an attribute for
runtime inspection:
def optional(f: _F) -> _F:
f.__optional__ = True
return f
Full support for type checkers is more complicated, as hasattr() support
would be required to check implementations. A quick & dirty way for type
checkers to support the status quo is to ignore @optional fields.
- Sebastian
Typing is hard. And constantly evolving. I've seen questions about
correct typing in many different forums and bug trackers, and often the
same questions get repeatedly asked. This is why we set up a central
forum to help users with typing at
https://github.com/python/typing/discussions. Currently, it's a bit bare
bones, but we have two categories for general Q&A and for asking for
reviews. If users have questions about typing, please feel free to
direct them there or to Gitter (https://gitter.im/python/typing). If you
maintain a typing-related package or typing-related documentation, we
would be happy if you could add a link to the discussions forum to your
documentation. Contributors to the forum and suggestions on how to
improve it are most welcome.
That forum is intended as a user help forum. Discussions about new
features, improvements, and implementation should continue to go to this
list or to the appropriate issue trackers.
- Sebastian
Hello typing-sig,
I wanted to suggest adding support for sealed classes to the python type
system.
What are sealed classes?
Sealed classes are classes of which all subclasses are known statically.
This is usually accomplished by requiring all subclasses to be created
in the same file/module as the parent class.
What languages support sealed classes?
Sealed classes are supported by C#, Kotlin and starting with Java 15
even by Java.
What can sealed classes be used for?
Sealed classes are useful for checking the exhaustiveness of if/else
towers and (python 3.10) match statements for example when working with
abstract syntax trees.
Take a look at this example:
```
class Node(ABC): ...
class Value(Node): ...
class BinOp(Node): ...
class UnOp(Node): ...
n: Node
match n:
case Value(v): ...
case BinOp(n1, op, n2): ...
case UnOp(op, n1): ...
# Can we statically check if all cases are handled?
```
Sealed classes in python:
My suggestion would be to add a new class "Sealed" to typing and
typing_extensions that can be inherited from:
class Node(Sealed): ...
Type checkers should make sure that no subclasses of Node are created in
modules other than the module Node was created in and should treat
sealed classes as abstract classes. Additionally they can keep track of
all the subclasses of Node do implement exhaustiveness checking.
The runtime implementation of sealed could also prevent non-typechecked
code from creating additional subclasses:
```
class Sealed(ABC):
def __init_subclass__(cls):
for c in cls.__mro__:
if Sealed in c.__bases__ and c.__module__ != cls.__module__:
raise RuntimeError('Sealed class can\'t be subclassed
from another module. Sealed in module "{}".'.format(c.__module__))
```
Final example:
a.py:
```
from typing import Sealed
class Base(Sealed):
pass
class A(Base):
pass
class B(A):
pass
```
b.py:
```
from a import Base, A
class C(Base): # raises an exception
pass
class D(A): # also raises an exception
pass
```
Adrian Freund
Hi,
Context: during the typing meeting of this week Brandon presented the idea of introducing a Compose type operator, which takes a variadic of callables, checks that they compose, and returns the type of the composed function.
To follow-up on the discussion about Compose:
We were wondering what can be achieved nowadays without custom operators, just relying on overload. Let's think about the example of a compose function, we could do the following right?
from typing import Callable as Call
# Overloads
def compose(f1: Call[A,B]) -> Call[A,B]
def compose(f1: Call[A,B], f2: Call[B,C]) -> Call[A,C]
def compose(f1: Call[A,B], f2: Call[B,C], f3: Call[C,D]) -> Call[A,D]
...
What is less obvious is how would we type a class like torch.nn.Sequential, which would look like this with the Compose operator:
class Sequential(Generic[A,*Ts]):
def __init__(self, fs: *Ts)
def __call__(self, in: A) -> Compose[*Ts]
Here we can't overload the class definition, so what can be done? We had a couple of ideas but they would not really work for a variable number of arguments because we can't overload. An example of those approaches would be:
class Sequential(Generic[A,B,C]):
def __init__(self, f1: Callable[A,B], f2: Callable[B,C]): ...
def __call__(self, in: A) -> C: ....
Best,
Alfonso.
Hi all,
Out next tensor typing meeting will be *Monday the 16th of August at 6pm UK
time/10am San Francisco time*. Our tentative agenda is:
1. PEP 646 progress (basically that grammar changes are done and we're
just waiting for review on updates to the PEP itself)
2. Discussion of feedback we got from TensorFlow/PyTorch core devs
Give me a shout if anyone's interested in talking about something else!
Cheers,
Matthew
I would like to draft a PEP for a typing.Self class which would perform similarly to rust's Self keyword in function annotations.
Examples
```python
from typing import TypeVar
F = TypeVar("F", bound="Foo")
class Foo:
def parse(self: F, data: bytes) -> F:
...
return self
```
vs
```python
from typing import Self
class Foo:
def parse(self, data: bytes) -> Self:
...
return self
```
Another example of where this could be useful is classmethods:
```python
class Foo:
@classmethod
def bar(cls: type[F], *args: Any, **kwargs: Any) -> F:
return cls()
```
vs
```python
class Foo:
@classmethod
def bar(cls, *args: Any, **kwargs: Any) -> Self:
return cls()
```
Implementation wise it would be a copy of NoReturn, i.e. not subscriptable, a _SpecialForm and should only be valid as a return type.
I haven't seen any discussion of the idea on the mailing list, so I wanted to run it by everyone.
Thanks,
James.
Hi typing-sig,
I’d like to start a discussion about moving forward with improved syntax for typing.Callable based on discussion in the Typing Summit:
* Allow `(ArgType0, ArgType1) -> ReturnType`
* In place of `Callable[[ArgType0, ArgType1], ReturnType]`
A lot of folks also want a way to express things like named, optional, and variadic args, but this is more complicated and has harder tradeoffs.
I'll start a separate thread next week to discuss that, let's aim to keep this thread focused on the syntactic sugar described above.
If we think this is a good idea, I'm happy to work on drafting a PEP.
## Background
The existing Callable type is useful for simple functions with required arguments passed by position, but there’s consensus that it can be hard to keep track of brackets, especially as the types get complex.
For example, even in very simple functional-style code with higher-order functions the brackets can be cumbersome:
```
from typing import Callable
def accepts_callback(callback: Callable[[str, int], str]) -> None:
print(callback("answer", 42)
def f(s: str, x: int) -> str:
return s + ": " + str(x)
accepts_callback(f)
accepts_callback(lambda s, _: s)
```
In agreement with a poll from the Typing Summit, I’d like to propose allowing
* `(Argtype1, Argtype2) -> ReturnType` to be used in place of
* `Callable[[Argtype1, Argtype2], ReturnType]`
This allows us to rewrite the example above as:
```
def accepts_callback(callback: (str, int) -> str) -> None:
print(callback("answer", 42)
def f(s: str, x: int) -> str:
return s + ": " + str(x)
accepts_callback(f)
accepts_callback(lambda s, _: s)
```
Advantages of arrow syntax include:
* eliminate the need to import Callable
* easier-to-skim types
* they are shorter, and one of the problems with Callables is just verbosity
* the parentheses and arrow have more visual structure than existing bracket-based Callable syntax
* a syntax that should feel familiar:
* it mimics the syntax for ordinary functions and stubs
* it is similar to several other languages e.g.
* [typescript](https://basarat.gitbook.io/typescript/type-system/callable#arrow-syntax)
* [flow](https://flow.org/en/docs/types/functions/#toc-function-types)
* [kotlin](https://kotlinlang.org/docs/lambdas.html#function-types)
## Open questions about syntax rules
Would we require parentheses around a callable return type as above, or allow it to be bare (which causes two ->s to be visually at the same level of the code, even though one is nested in the AST)?
Consider this function (annotated using a conservative syntax where we wrap the return type in paranetheses):
```
def identity(
f: (str, int, int) -> int,
) -> ((str, int, int) -> int):
return f
```
In this case, would we want to
* require parentheses around a callable return type as above, or
* allow it to be bare (which causes two ->s to look a little bit as if they are at the same code level)
Here’s what the example above would look like if we allow bare arrow syntax in the return type:
```
def identity(
f: (str, int, int) -> int,
) -> (str, int, int) -> int:
return f
```
Another option, if we feel like the outer parentheses are necessary would be to make the parameter grouping optional in this case, for example:
```
def identity(
f: (str, int, int) -> int,
) -> (str, int, int -> int):
return f
```
## Alternatives considered
We ran a poll at the typing summit comparing three options for nicer Callable syntax:
* This proposal `(ArgType1, ArgType2, ArgType3) -> ReturnType`
* 72% of participants upvoted this
* A typescript-like syntax `(a: ArgType1, b: ArgType2) => ReturnType`
* 62% of participants upvoted this
* Disadvantages are
* a new symbol `=>`
* argument names are nuisance parameters for positional args (which is all the existing Callable supports)
* `[ArgType1, ArgType2, ArgType3] -> ReturnType`
* 8% of participants upvoted this
* The main disadvantage is that brackets don’t seem as intuitive
- Steven
