pt., 8 paź 2021 o 04:48 S Pradeep Kumar napisał(a):

... Note that we considered and rejected using a full def-signature syntax like ```` (record: PurchaseRecord, permissions: List[AuthPermission], /) -> FormattedItem ```` because it would be more verbose for common cases and could lead to subtle bugs; more details in [3].

Did you considered full def-signature with optional argument names, so the common cases would look like ```` (:PurchaseRecord, :List[AuthPermission]) -> FormattedItem ```` Bare name signatures like '(record) -> FormattedItem' could be disallowed to prevent bugs.

On Fri, 8 Oct 2021 at 03:45, S Pradeep Kumar wrote:

Typing-sig has been discussing user-friendly syntax for the type used to represent callables. [1] Since this affects the Python language syntax, we wanted to get some high-level feedback from you before putting up a detailed PEP.

Disclaimer: I personally find that use of complex type annotations like Callable make code unreadable for the *human* reader, as compared to out of line information like a clearly-written comment or documentation. Therefore I'd probably be extremely reluctant to use this new syntax in real-life projects, regardless of the form that it takes. However, I do have some experience working on projects that use type annotations like this for callbacks, so I will answer in spite of my personal reservations.

**Question 1**: Are there concerns we should keep in mind about such a syntax proposal?

In my experience, a common pattern for functions that take non-trivial callbacks is to call them with named arguments rather than positional, in order to preserve future compatibility. So I would like callback annotations to be able to express the type "a function returning X, taking arguments `arg1` of type Y and `arg2` of type Z". The lack of keyword argument support in this proposal makes it unusable for APIs like this. Conversely, for callbacks with single arguments like f(cb: int->int) -> int, in my experience a lot of uses are lambda expressions, so something like f(lambda x: x+1). I would be a strong -1 on having to add types to lambda expressions (they are ugly enough already) so how does this interact with the annotation? Are type checkers able to correctly infer the type of lambda x: x+1, or would the whole expression end up being untyped? Similarly with functools.partial - would that correctly match the type? If common uses of callbacks end up being untyped in practice, that drastically reduces the value of the new syntax. (Note that this point is equally applicable to Callable - so I guess it's not so much about the new syntax as it is about whether annotating callbacks is sufficiently useful to be *worth* improving). Also, note that I automatically used a type of int->int up there. As someone else asked, is that form allowed, or is it required to be (int)->int? In my view, if we require the latter, I expect there will be a *lot* of people making that mistake and having to correct it.

**Question 2**: Do you have preferences either way? Do we propose (1) alone or (1) + (2)?

See above. My experience is that using named arguments for callbacks is a useful approach, and should probably be encouraged rather than made difficult, so I'd say that named argument support is important. Having said that, I think that making simple types like int->int or (int, int)->int more awkward to write would be a disadvantage - they are used a *lot* in things like the `key` argument of `sort`. So I'm -1 on things like (for example) requiring a colon as in (:int, :int) -> int. But to reiterate, I'm probably not likely to ever be a significant consumer of this feature, so consider the above with that in mind ;-) Paul

El vie, 8 oct 2021 a las 0:54, Paul Moore () escribió:

On Fri, 8 Oct 2021 at 03:45, S Pradeep Kumar wrote:

...

Typing-sig has been discussing user-friendly syntax for the type used to represent callables. [1] Since this affects the Python language syntax, we wanted to get some high-level feedback from you before putting up a detailed PEP.

Disclaimer: I personally find that use of complex type annotations like Callable make code unreadable for the *human* reader, as compared to out of line information like a clearly-written comment or documentation. Therefore I'd probably be extremely reluctant to use this new syntax in real-life projects, regardless of the form that it takes. However, I do have some experience working on projects that use type annotations like this for callbacks, so I will answer in spite of my personal reservations.

The point of this change is to make Callable annotations more readable: (int) -> int is a lot less dense than Callable[[int], int]. We hope that the new syntax will make it easier to use more complex annotations. Of course, if you think the annotations are still too complex, that's your choice.

...

**Question 1**: Are there concerns we should keep in mind about such a syntax proposal?

In my experience, a common pattern for functions that take non-trivial callbacks is to call them with named arguments rather than positional, in order to preserve future compatibility. So I would like callback annotations to be able to express the type "a function returning X, taking arguments `arg1` of type Y and `arg2` of type Z". The lack of keyword argument support in this proposal makes it unusable for APIs like this.

That's one of the differences between the proposals discussed here. The basic proposal Pradeep pointed out would not support named arguments, the more complete syntax favored by Guido (and also by me; 2(a) in Pradeep's email) would support it. Pradeep has done some empirical analysis that shows that in practice it is not very common to use such types, though.

Conversely, for callbacks with single arguments like f(cb: int->int) -> int, in my experience a lot of uses are lambda expressions, so something like f(lambda x: x+1). I would be a strong -1 on having to add types to lambda expressions (they are ugly enough already) so how does this interact with the annotation? Are type checkers able to correctly infer the type of lambda x: x+1, or would the whole expression end up being untyped? Similarly with functools.partial - would that correctly match the type? If common uses of callbacks end up being untyped in practice, that drastically reduces the value of the new syntax. (Note that this point is equally applicable to Callable - so I guess it's not so much about the new syntax as it is about whether annotating callbacks is sufficiently useful to be *worth* improving).

Type checkers should be able to infer this type. You would not need to write any explicit type when you call f(). However, the definition of `f` would look something like `def f(callback: (int) -> int) -> None: ...`. Currently, you'd write that as `def f(callback: Callable[[int], int]) -> None: ...`.

Also, note that I automatically used a type of int->int up there. As someone else asked, is that form allowed, or is it required to be (int)->int? In my view, if we require the latter, I expect there will be a *lot* of people making that mistake and having to correct it.

That's not something we discussed before. I'd be OK with allowing this unless it makes the grammar too ambiguous.

...

**Question 2**: Do you have preferences either way? Do we propose (1) alone or (1) + (2)?

See above. My experience is that using named arguments for callbacks is a useful approach, and should probably be encouraged rather than made difficult, so I'd say that named argument support is important. Having said that, I think that making simple types like int->int or (int, int)->int more awkward to write would be a disadvantage - they are used a *lot* in things like the `key` argument of `sort`. So I'm -1 on things like (for example) requiring a colon as in (:int, :int) -> int.

But to reiterate, I'm probably not likely to ever be a significant consumer of this feature, so consider the above with that in mind ;-)

Paul _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ANGLWMP4... Code of Conduct: http://python.org/psf/codeofconduct/

Thanks for the responses! 1. Chris Angelico:

it looks like the parentheses are mandatory, and they are what define it?

Yes, we always expect parens. Like you said, this keeps things consistent, resembles the function signature, and avoids ambiguities. It also won't look like Haskell function types `int -> str -> bool` (which can be confusing for non-functional programmers). This was pretty explicitly rejected when we polled people at the Typing Summit.

Which of these will be valid?

(int) -> int (int, int) -> int (int,) -> int should also be fine, but I don't have strong opinions here. e.g., what about (,) -> int? We'll make sure to address this edge case explicitly, thanks.

There are parallel proposals to support a lambda syntax like "x => x.spam" as equivalent to "lambda x: x.spam", so I'm curious what the rules would be for the two types of arrows, and whether they'd be consistent with each other.

Did you considered full def-signature with optional argument names, so

Yeah, this had come up in our discussions. Like in other languages (such as Hack), we believe that lambdas should have a different arrow type to disambiguate the two. So, lambdas could be (x, y) => x + y. 2. Piotr Duda the common cases would look like

```` (:PurchaseRecord, :List[AuthPermission]) -> FormattedItem ```` Bare name signatures like '(record) -> FormattedItem' could be disallowed

to prevent bugs. The additional-features proposal in (2) actually does have optional parameter names. There are two variants: (a) the "hybrid" proposal, which will let you write (int, str) -> bool, but also hybrids like (int, y: str) -> bool; (x: int, y: str) -> bool; (int, str=...) -> bool. (b) the "xor" proposal, which will let you use the shorthand syntax (int, str) -> bool for the most frequently-used cases and the full def-signature for all other advanced cases using familiar syntax: (x: int, y: str=...) -> bool. We haven't reached a strong consensus on the above two yet (flexibility vs familiarity). We decided to get some thoughts here about whether we need to propose additional-features at all (2) in our initial PEP. 3. Patrick Reader

How would this work for a function that returns a function? Would you just put it on the def line like this?

We will allow using the callable syntax in the return type without parens: def foo() -> (int) -> str: return int_to_str The reason for not making that a syntax error is that it would be pretty bad UX. Users would see the callable syntax being used without parens everywhere: f: (int) -> str, but when they try to use that as the return type, they would get a confusing syntax error. So, given how frequently functions return functions, we decided to not require parens. People can specify parens in their coding guidelines (and auto-formatters) if they find parens more readable. 4. Paul Moore +1 to what Jelle said.

That's one of the differences between the proposals discussed here. The basic proposal Pradeep pointed out would not support named arguments, the more complete syntax favored by Guido (and also by me; 2(a) in Pradeep's email) would support it.

Pradeep has done some empirical analysis that shows that in practice it is not very common to use such types, though.

Yeah, that's basically Question 2 - whether to just replace Callable in the initial PEP (option 1) or to specify a more complete syntax from the beginning (option 2). 5. Serhiy Storchaka

How could you replace Callable[..., int] and Callable[Concatenate[str, P], int] ?

To represent a Callable that accepts arbitrary arguments: (...) -> int To represent a Callable that accepts ParamSpec (for decorators, etc.), we're leaning towards the below: (**P) -> int To represent your Concatenate example: (str, **P) -> int We would no longer need the Concatenate operator. We can naturally just add types up front. -- S Pradeep Kumar On Fri, Oct 8, 2021 at 8:48 AM Guido van Rossum wrote:

On Fri, Oct 8, 2021 at 8:31 AM Jelle Zijlstra wrote:

...

El vie, 8 oct 2021 a las 0:54, Paul Moore () escribió:

...

Also, note that I automatically used a type of int->int up there. As someone else asked, is that form allowed, or is it required to be (int)->int? In my view, if we require the latter, I expect there will be a *lot* of people making that mistake and having to correct it.

That's not something we discussed before. I'd be OK with allowing this unless it makes the grammar too ambiguous.

Even if it didn't make the grammar ambiguous (and I think it doesn't, as long as the argument type isn't a union or another callable type), I'd be against this.

An argument list is not a tuple, and we don't allow omitting the parentheses in other call-related situations: you can't write "def f x: return x+1" and you can't write "f 42". Allowing the omission of the parentheses here would be inconsistent (even if some other languages allow it).

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

-- S Pradeep Kumar

Pradeep, Thanks for all the clarifications. I think it's time that you put together a formal grammar. That should also make clear that "(,) -> int" is disallowed while "(int,) -> int" is allowed. --Guido On Fri, Oct 8, 2021 at 8:50 AM S Pradeep Kumar wrote:

Thanks for the responses!

1. Chris Angelico:

...

it looks like the parentheses are mandatory, and they are what define it?

Yes, we always expect parens. Like you said, this keeps things consistent, resembles the function signature, and avoids ambiguities. It also won't look like Haskell function types `int -> str -> bool` (which can be confusing for non-functional programmers).

This was pretty explicitly rejected when we polled people at the Typing Summit.

...

Which of these will be valid?

(int) -> int (int, int) -> int

(int,) -> int should also be fine, but I don't have strong opinions here. e.g., what about (,) -> int? We'll make sure to address this edge case explicitly, thanks.

...

There are parallel proposals to support a lambda syntax like "x => x.spam" as equivalent to "lambda x: x.spam", so I'm curious what the rules would be for the two types of arrows, and whether they'd be consistent with each other.

Yeah, this had come up in our discussions. Like in other languages (such as Hack), we believe that lambdas should have a different arrow type to disambiguate the two. So, lambdas could be (x, y) => x + y.

2. Piotr Duda

...

Did you considered full def-signature with optional argument names, so the common cases would look like

```` (:PurchaseRecord, :List[AuthPermission]) -> FormattedItem ```` Bare name signatures like '(record) -> FormattedItem' could be disallowed to prevent bugs.

The additional-features proposal in (2) actually does have optional parameter names. There are two variants:

(a) the "hybrid" proposal, which will let you write (int, str) -> bool, but also hybrids like (int, y: str) -> bool; (x: int, y: str) -> bool; (int, str=...) -> bool.

(b) the "xor" proposal, which will let you use the shorthand syntax (int, str) -> bool for the most frequently-used cases and the full def-signature for all other advanced cases using familiar syntax: (x: int, y: str=...) -> bool.

We haven't reached a strong consensus on the above two yet (flexibility vs familiarity). We decided to get some thoughts here about whether we need to propose additional-features at all (2) in our initial PEP.

3. Patrick Reader

...

How would this work for a function that returns a function? Would you just put it on the def line like this?

We will allow using the callable syntax in the return type without parens:

def foo() -> (int) -> str: return int_to_str

The reason for not making that a syntax error is that it would be pretty bad UX. Users would see the callable syntax being used without parens everywhere: f: (int) -> str, but when they try to use that as the return type, they would get a confusing syntax error. So, given how frequently functions return functions, we decided to not require parens.

People can specify parens in their coding guidelines (and auto-formatters) if they find parens more readable.

4. Paul Moore

+1 to what Jelle said.

...

That's one of the differences between the proposals discussed here. The basic proposal Pradeep pointed out would not support named arguments, the more complete syntax favored by Guido (and also by me; 2(a) in Pradeep's email) would support it.

Pradeep has done some empirical analysis that shows that in practice it is not very common to use such types, though.

Yeah, that's basically Question 2 - whether to just replace Callable in the initial PEP (option 1) or to specify a more complete syntax from the beginning (option 2).

5. Serhiy Storchaka

...

How could you replace Callable[..., int] and Callable[Concatenate[str, P], int] ?

To represent a Callable that accepts arbitrary arguments:

(...) -> int

To represent a Callable that accepts ParamSpec (for decorators, etc.), we're leaning towards the below:

(**P) -> int

To represent your Concatenate example:

(str, **P) -> int

We would no longer need the Concatenate operator. We can naturally just add types up front. -- S Pradeep Kumar

On Fri, Oct 8, 2021 at 8:48 AM Guido van Rossum wrote:

...

On Fri, Oct 8, 2021 at 8:31 AM Jelle Zijlstra wrote:

...

El vie, 8 oct 2021 a las 0:54, Paul Moore () escribió:

...

Also, note that I automatically used a type of int->int up there. As someone else asked, is that form allowed, or is it required to be (int)->int? In my view, if we require the latter, I expect there will be a *lot* of people making that mistake and having to correct it.

That's not something we discussed before. I'd be OK with allowing this unless it makes the grammar too ambiguous.

Even if it didn't make the grammar ambiguous (and I think it doesn't, as long as the argument type isn't a union or another callable type), I'd be against this.

An argument list is not a tuple, and we don't allow omitting the parentheses in other call-related situations: you can't write "def f x: return x+1" and you can't write "f 42". Allowing the omission of the parentheses here would be inconsistent (even if some other languages allow it).

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

-- S Pradeep Kumar

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Thanks for summarizing the PSC position, Barry!

I agree that type annotation syntax should be consistent with the rest of
the language, but I'm curious why it has to be "standard Python syntax"?
Could you elaborate a bit more on the reasoning behind that constraint?

Cheers,
Sergei

On Fri, Oct 8, 2021 at 8:38 PM Barry Warsaw  wrote:

> Thanks for thinking more deeply about how we can make type annotations for
> callables more user friendly.
>
> The intersection between the syntax for general Python code and type
> annotations has been a topic of discussion within the Steering Council (on
> and off) for quite some time.  We intend to provide more clarity about our
> thinking, probably in the form of an informational PEP, around this topic,
> and my apologies because I haven’t had time to draft it yet.
>
> But since this came up, I want to at least say that we are unanimous in
> our belief that the type annotation language must also be standard Python
> syntax.  In other words, we are not in favor of letting type annotation
> syntax deviate from Python syntax.  As a effect of this, the PSC does not
> intend to issue a blanket delegation of type-focused PEPs.  Of course,
> individual typing PEPs can be delegated just like any other PEP, but
> especially where syntax change proposals are made, I expect that the PSC
> will want keep final decision making.
>
> This means that any PEP which proposes syntactic changes to support typing
> features must also address the implications of those syntax changes for the
> general Python language.  PEP 646 (Variadic Generics) is a good example of
> this.  Early on we recognized that the proposed syntax change had
> implications for Python, and we asked the PEP authors to address those
> implications for Python, which they added:
>
> https://www.python.org/dev/peps/pep-0646/#grammar-changes
>
> This is the kind of analysis the PSC needs in order to weigh the cost and
> benefits of making such changes to Python.
>
> TL;RA - One Syntax to Rule Them All
>
> Cheers,
> -Barry
>
> > On Oct 7, 2021, at 09:41, S Pradeep Kumar  wrote:
> >
> > Hello all,
> >
> > Typing-sig has been discussing user-friendly syntax for the type used to
> represent callables. [1] Since this affects the Python language syntax, we
> wanted to get some high-level feedback from you before putting up a
> detailed PEP.
> >
> > TL;DR: We want to propose syntax for Callable types, e.g., `(int, str)
> -> bool` instead of `typing.Callable[[int, str], bool]`. Questions: 1. Are
> there concerns we need to keep in mind about such a syntax change? 2.
> Should we propose syntax for additional features in the same PEP or in a
> future PEP?
> >
> >
> > # Motivation
> >
> > Guido has pointed out that `Callable` is one of the most frequently used
> type forms but has the least readable syntax. For example, say we have a
> callback `formatter` that accepts a record and a list of permissions:
> >
> > ```python
> > def print_purchases(
> >     user,
> >     formatter,  # <-- callback
> > ):
> >     <...>
> >     output = formatter(record, permissions)
> >     print(output)
> > ```
> >
> > To give it a type, we currently have to write:
> >
> > ```python
> > from typing import Callable
> >
> > def print_purchases(
> >     user: User,
> >     formatter: Callable[[PurchaseRecord, List[AuthPermission]],
> FormattedItem]  # Hard to read.
> > ) -> None:
> >
> >     <...>
> >     output = formatter(record, permissions)
> >     print(output)
> > ```
> >
> > `Callable` can be hard to understand for new users since it doesn't
> resemble existing function signatures and there can be multiple square
> brackets. It also requires an import from `typing`, which is inconvenient.
> Around 40% of the time [2], users just give up on precisely typing the
> parameters and return type of their callbacks and just leave it as a blank
> Callable. In other cases, they don't add a type for their callback at all.
> Both mean that they lose the safety guarantees from typing and leave room
> for bugs.
> >
> > We believe that adding friendly syntax for Callable types will improve
> readability and encourage users to type their callbacks more precisely.
> Other modern, gradually-typed languages like TypeScript (JS), Hack (PHP),
> etc. have special syntax for Callable types.
> >
> > (As a precedent, PEP 604 recently added clean, user-friendly syntax for
> the widely-used `Union` type. Instead of importing `Union` and writing
> `expr: Union[AddExpr, SubtractExpr], we can just write `expr: AddExpr |
> SubtractExpr`.)
> >
> >
> > # Proposal and Questions
> >
> > We have a two-part proposal and questions for you:
> >
> > 1. Syntax to replace Callable
> >
> > After a lot of discussion, there is strong consensus in typing-sig about
> adding syntax to replace Callable. So, the above example would be written
> as:
> > ```python
> > def print_purchases(
> >     user: User,
> >     formatter: (PurchaseRecord, List[AuthPermission]) -> FormattedItem,
> > ) -> None:
> >
> >     <...>
> >     output = formatter(record, permissions)
> >     print(output)
> > ```
> > This feels more natural and succinct.
> >
> > Async callbacks currently need to be written as
> > ```
> > from typing import Callable
> >
> > async_callback: Callable[[HttpRequest], Awaitable[HttpResponse]]
> > ```
> >
> > With the new syntax, they would be written as
> > ```
> > async_callback: async (HttpRequest) -> HttpResponse
> > ```
> > which again seems more natural. There is similar syntax for the type of
> decorators that pass on *args and **kwargs to the decorated function.
> >
> > Note that we considered and rejected using a full def-signature syntax
> like
> > ````
> > (record: PurchaseRecord, permissions: List[AuthPermission], /) ->
> FormattedItem
> > ````
> > because it would be more verbose for common cases and could lead to
> subtle bugs; more details in [3].
> >
> > The Callable type is also usable as an expression, like in type aliases
> `IntOperator = (int, int) -> int` and `cast((int) -> int, f)` calls.
> >
> > **Question 1**: Are there concerns we should keep in mind about such a
> syntax proposal?
> >
> >
> >
> > 2. Syntax for callback types beyond Callable
> >
> > `Callable` can't express the type of all possible callbacks. For
> example, it doesn't support callbacks where some parameters have default
> values: `formatter(record)` (the user didn't pass in `permissions`). It
> *is* possible to express these advanced cases using Callback Protocols (PEP
> 544) [4] but it gets verbose.
> >
> > There are two schools of thought on typing-sig on adding more syntax on
> top of (1):
> >
> > (a) Some, including Guido, feel that it would be a shame to not have
> syntax for core Python features like default values, keyword arguments, etc.
> >
> >     One way to represent default values or optionally name parameters
> would be:
> >
> >     ```
> >     # permissions is optional
> >     formatter: (PurchaseRecord, List[AuthPermission]=...) ->
> FormattedItem
> >
> >     # permissions can be called using a keyword argument.
> >     formatter: (PurchaseRecord, permissions: List[AuthPermission]) ->
> FormattedItem
> >     ```
> >
> >     There are also alternative syntax proposals.
> >
> > (b) Others want to wait till we have more real-world experience with the
> syntax in (1).
> >
> >     The above cases occur < 5% of the time in typed or untyped code [5].
> And the syntax in (1) is forward-compatible with the additional proposals.
> So we could add them later if needed or leave them out, since we can always
> use callback protocols.
> >
> > **Question 2**: Do you have preferences either way? Do we propose (1)
> alone or (1) + (2)?
> >
> >
> > Once we get some high-level feedback here, we will draft a PEP going
> into the details for various use cases.
> >
> > Best,
> > Pradeep Kumar Srinivasan
> > Steven Troxler
> > Eric Traut
> >
> > PS: We've linked to more details below. Happy to provide more details as
> needed.
> >
> > [1]: typing-sig thread about the proposal:
> https://mail.python.org/archives/list/typing-sig@python.org/message/JZLYRAXJV34WAV5TKEOMA32V7ZLPOBFC/
> > [2]: Stats about loosely-typed Callables:
> https://github.com/pradeep90/annotation_collector#typed-projects---callable-type
> > [3]: Comparison and rejection of proposals:
> https://www.dropbox.com/s/sshgtr4p30cs0vc/Python%20Callable%20Syntax%20Proposals.pdf?dl=0
> > [4]: Callback protocols:
> https://www.python.org/dev/peps/pep-0544/#callback-protocols
> > [5]: Stats on callbacks not expressible with Callable:
> https://drive.google.com/file/d/1k_TqrNKcbWihRZdhMGf6K_GcLmn9ny3m/view?usp=sharing
> > _______________________________________________
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-leave@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/VBHJOS3LOXGVU6I4FABM6DKHH65GGCUB/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
> _______________________________________________
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-leave@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/4TY3MVJQOLKSTMCJRTGAZEOSIIMAWQ45/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

On Fri, Oct 8, 2021 at 3:36 PM Barry Warsaw wrote:

On Oct 8, 2021, at 13:02, Sergei Lebedev wrote:

...

I agree that type annotation syntax should be consistent with the rest

of the language, but I'm curious why it has to be "standard Python syntax"? Could you elaborate a bit more on the reasoning behind that constraint?

Hi Sergei. I don’t mean anything more than just that there should be a single syntax for all of Python. I was just trying to describe “the bits of Python that aren’t type annotations”.

I interpret this as follows: - Whatever we choose (say, "(int, int) -> int") should be allowed where expressions are allowed, so that e.g. foo = (int, int) -> int assigns *something* at runtime -- it shouldn't be a syntax error and it shouldn't raise an exception. My preference would be for it to return something that's introspectable, i.e. an object lets one recover the original info. That's what we did when we introduced "A | B" as union syntax for types. (It's slightly different because that was already valid syntax, but the principle that it must return something introspectable is the same.) Union objects have an `__args__` attribute that gives the underlying types, and a `__parameters__` attribute giving any type variables. Note that the implementation of this introspectable type should ultimately be in C. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

We drew up a draft syntax to clarify the three styles that have been discussed in typing-sig: - shorthand syntax, which supports exactly what Callable currently supports - XOR syntax, which would allow a def-like style in addition to shorthand - hybrid syntax, which extends shorthand to support named, default-value, and variadic args https://docs.google.com/document/d/1oCRBAAPs3efTYoIRSUwWLtOr1AUVqc8OCEYn4vKY... Within the discussion in typing-sig, - everyone would like to see support for at least shorthand - a little over half would like to have the more complex hybrid option - I don't think anyone has XOR as their first choice, but several of us think it might be better than hybrid

On 11/10/2021 17:03, Steven Troxler wrote:

Personally, I think we should both adopt function-as-a-type and shorthand syntax, and reject both the XOR and hybrid syntax because function-as-a-type is convenient for expressing the same things (named, default-value, and variadic args).

+1 This combination of a shorthand syntax for simple callbacks and a longhand interface-style method for more complex signatures is the best option IMO. It provides good flexibility, in the pragmatic sense of "there should be only one way to do it": there is a dead-simple way which is pretty good, and a more flexible way for more complex cases which is pretty easy to switch to if necessary.

If I can make a wild suggestion: why not create a little language for type specifications? If you look at other programming languages you’ll see that the “type definition sub-language” is often completely different from the “execution sub-language”, with only some symbols in common and used in vaguely related ways. `bool (*myfuncptr)(int, float*)` uses a completely different set of syntactic rules than `rv = (*myfunptr)(*myintptr, &myfloat)`. So with some grains of salt you could say that C is comprised of a declarative typing sublanguage and an imperative execution sublanguage. Python typing uses basically a subset of the execution expression syntax as its declarative typing language. What if we created a little language that is clearly flagged, for example as t”….” or t’….’? Then we could simply define the typestring language to be readable, so you could indeed say t”(int, str) -> bool”. And we could even allow escapes (similar to f-strings) so that the previous expression could also be specified, if you really wanted to, as t”{typing.Callable[[int, str], bool}”. Jack

On 7 Oct 2021, at 18:41, S Pradeep Kumar wrote:

Hello all,

Typing-sig has been discussing user-friendly syntax for the type used to represent callables. [1] Since this affects the Python language syntax, we wanted to get some high-level feedback from you before putting up a detailed PEP.

TL;DR: We want to propose syntax for Callable types, e.g., `(int, str) -> bool` instead of `typing.Callable[[int, str], bool]`. Questions: 1. Are there concerns we need to keep in mind about such a syntax change? 2. Should we propose syntax for additional features in the same PEP or in a future PEP?

# Motivation

Guido has pointed out that `Callable` is one of the most frequently used type forms but has the least readable syntax. For example, say we have a callback `formatter` that accepts a record and a list of permissions:

```python def print_purchases( user, formatter, # <-- callback ): <...> output = formatter(record, permissions) print(output) ```

To give it a type, we currently have to write:

```python from typing import Callable

def print_purchases( user: User, formatter: Callable[[PurchaseRecord, List[AuthPermission]], FormattedItem] # Hard to read. ) -> None:

<...> output = formatter(record, permissions) print(output) ```

`Callable` can be hard to understand for new users since it doesn't resemble existing function signatures and there can be multiple square brackets. It also requires an import from `typing`, which is inconvenient. Around 40% of the time [2], users just give up on precisely typing the parameters and return type of their callbacks and just leave it as a blank Callable. In other cases, they don't add a type for their callback at all. Both mean that they lose the safety guarantees from typing and leave room for bugs.

We believe that adding friendly syntax for Callable types will improve readability and encourage users to type their callbacks more precisely. Other modern, gradually-typed languages like TypeScript (JS), Hack (PHP), etc. have special syntax for Callable types.

(As a precedent, PEP 604 recently added clean, user-friendly syntax for the widely-used `Union` type. Instead of importing `Union` and writing `expr: Union[AddExpr, SubtractExpr], we can just write `expr: AddExpr | SubtractExpr`.)

# Proposal and Questions

We have a two-part proposal and questions for you:

1. Syntax to replace Callable

After a lot of discussion, there is strong consensus in typing-sig about adding syntax to replace Callable. So, the above example would be written as: ```python def print_purchases( user: User, formatter: (PurchaseRecord, List[AuthPermission]) -> FormattedItem, ) -> None:

<...> output = formatter(record, permissions) print(output) ``` This feels more natural and succinct.

Async callbacks currently need to be written as ``` from typing import Callable

async_callback: Callable[[HttpRequest], Awaitable[HttpResponse]] ```

With the new syntax, they would be written as ``` async_callback: async (HttpRequest) -> HttpResponse ``` which again seems more natural. There is similar syntax for the type of decorators that pass on *args and **kwargs to the decorated function.

Note that we considered and rejected using a full def-signature syntax like ```` (record: PurchaseRecord, permissions: List[AuthPermission], /) -> FormattedItem ```` because it would be more verbose for common cases and could lead to subtle bugs; more details in [3].

The Callable type is also usable as an expression, like in type aliases `IntOperator = (int, int) -> int` and `cast((int) -> int, f)` calls.

**Question 1**: Are there concerns we should keep in mind about such a syntax proposal?

2. Syntax for callback types beyond Callable

`Callable` can't express the type of all possible callbacks. For example, it doesn't support callbacks where some parameters have default values: `formatter(record)` (the user didn't pass in `permissions`). It *is* possible to express these advanced cases using Callback Protocols (PEP 544) [4] but it gets verbose.

There are two schools of thought on typing-sig on adding more syntax on top of (1):

(a) Some, including Guido, feel that it would be a shame to not have syntax for core Python features like default values, keyword arguments, etc.

One way to represent default values or optionally name parameters would be:

``` # permissions is optional formatter: (PurchaseRecord, List[AuthPermission]=...) -> FormattedItem

# permissions can be called using a keyword argument. formatter: (PurchaseRecord, permissions: List[AuthPermission]) -> FormattedItem ```

There are also alternative syntax proposals.

(b) Others want to wait till we have more real-world experience with the syntax in (1).

The above cases occur < 5% of the time in typed or untyped code [5]. And the syntax in (1) is forward-compatible with the additional proposals. So we could add them later if needed or leave them out, since we can always use callback protocols.

**Question 2**: Do you have preferences either way? Do we propose (1) alone or (1) + (2)?

Once we get some high-level feedback here, we will draft a PEP going into the details for various use cases.

Best, Pradeep Kumar Srinivasan Steven Troxler Eric Traut

PS: We've linked to more details below. Happy to provide more details as needed.

[1]: typing-sig thread about the proposal: https://mail.python.org/archives/list/typing-sig@python.org/message/JZLYRAXJ... https://mail.python.org/archives/list/typing-sig@python.org/message/JZLYRAXJ... [2]: Stats about loosely-typed Callables: https://github.com/pradeep90/annotation_collector#typed-projects---callable-... https://github.com/pradeep90/annotation_collector#typed-projects---callable-... [3]: Comparison and rejection of proposals: https://www.dropbox.com/s/sshgtr4p30cs0vc/Python%20Callable%20Syntax%20Propo... https://www.dropbox.com/s/sshgtr4p30cs0vc/Python%20Callable%20Syntax%20Propo... [4]: Callback protocols: https://www.python.org/dev/peps/pep-0544/#callback-protocols https://www.python.org/dev/peps/pep-0544/#callback-protocols [5]: Stats on callbacks not expressible with Callable: https://drive.google.com/file/d/1k_TqrNKcbWihRZdhMGf6K_GcLmn9ny3m/view?usp=s... https://drive.google.com/file/d/1k_TqrNKcbWihRZdhMGf6K_GcLmn9ny3m/view?usp=s..._______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VBHJOS3L... Code of Conduct: http://python.org/psf/codeofconduct/

It isn't clear to me that reusing the inspect.Signature object was even considered. If it was rejected as too complicated for most signatures, please put that in the PEP. My own opinion, admittedly not as a typing fan, is that if you need enough details to want more than Callable, then you probably do want the full power of inspect.Signature, though you may also want some convenience methods to indicate that certain portions of the signature are undefined, rather than following the example function. -jJ

Thanks for the responses, everyone. Overall, it seems like there were no strong objections to the proposal. I didn't hear much about Question 2, though: Should we propose features beyond present-day `Callable` in the same PEP or defer it to a future PEP? In case that question got lost in the other details, feel free to respond here. If not, I'll take it there aren't strong opinions either way. Some of my other takeaways: + Address the implications of the syntax changes for Python. + Address edge cases like trailing commas, `Concatenate` for `ParamSpec`, and runtime value of the expression. + Explicitly discuss the function-name-as-a-type proposal. We will be drafting the PEP over the coming month.

12.10.21 00:03, Guido van Rossum пише:

But if I see

    def Comparison(a: T, b: T) -> Literal[-1, 0, 1]:         ...

my first thought is that it's a comparison function that someone hasn't finished writing yet, not a function type -- since if it did have at least one line of code in the body, it *would* be that.

I agree. There are also problems with variables substitution (indexing), union (operator "|") and introspection. We would need to add __getitem__, __or__, __args__, __parameters__ etc to all functions. There may be also conflicts with NewType which returns a callable. But can it be expressed via Protocol? class Comparison(Protocol, Generic[T]): def __call__(self, /, a: T, b: T) -> Literal[-1, 0, 1]: ... Then we can just add a decorator which turns a function into class: @functype def Comparison(a: T, b: T) -> Literal[-1, 0, 1]: ...