Type assignment statement (and extending the annotation syntax)
Many people seem to agree that Python typing still has some rough edges. Various PEPs have fixed warts (or proposed fixes), including PEPs 585, 563, and 649. Still, several issues remain: 1. Type annotation syntax is not always elegant (consider callable types). There's no good way to add custom type syntax without "polluting" non-typing use cases. 2. String literal escaping is ugly, and it's still needed from time to time. 3. Runtime introspection of types feels like a second-class citizen, even though it's pretty popular (pydantic, etc.). Instead of trying to tackle these issues piecemeal, I've tried to come up with ideas to solve all of the above issues. They don't form a complete proposal, and I haven't worked out many details yet, but I'm hoping that this could be a discussion starter, and perhaps it can help us eventually design something that is workable. Here I explain my first idea -- type assignment. Right now, we can't easily add dedicated syntax for callable types, for example, without either making this syntax available everywhere, or making it *un*available in type aliases and other common use cases, which don't use the annotation syntax. Supporting new syntax in annotations seems manageable: def doit(f: () => int) -> None: ... # Could be supported However, what about type aliases: A = () => int # Hmm? If this works, nothing stops people from writing weird code like this: d = {'x': (1 + y) => foo(1, 2)} I'd rather have dedicated typing syntax reserved for type-related uses, mainly to avoid confusing Python users who don't use typing. If typing syntax is only available via a small number of special syntactic forms, it's easier to teach users to ignore it, if they don't care about it. I'm proposing a new construct called "type assignment" where the RHS is lazily evaluated, and in the future it might support typing-only syntax. There would be two variants. First, we can define a type alias like this: type A = list[int] If we'd use the new syntax for callable types, we could write an alias like this: type C = (int) => str # No problem However, the following would be a syntax error, since it's not in a "type context": C = (int) => str # Error, as it probably should be Since the right hand side of type assignment contains types, we can avoid evaluating them during module import time, using either PEP 563 semantics (stringification) or something resembling PEP 649 (lazy evaluation). This means that we won't need to use explicit string literal escaping: type A = B # OK! class B: ... If the RHS is a function call, we'd call the function at runtime, but arguments would be (partially) stringified/lazy and they can contain type syntax (I'm glossing over some details): type N = NewType(int) type T = TypeVar(bound=(int) => str) # Type-specific syntax is ok type D = typing.TypedDict({"x": C}) # C can be a forward reference! class C: ... Note that the results here are usable in runtime contexts, since we call the function: x = N(2) # Ok The general syntax would be "type <id> = <name>(...)". We'd pass the stringified name as an implicit first argument for a cleaner syntax. We can write the earlier examples today like this: T = TypeVar("T", bound=Callable[[int], str]) N = NewType("N", int) D = typing.TypedDict("D", {"x": "C"}) # Need escaping here class C: ... The new syntax uses 'type', which is widely used as a name. The new meaning would only apply in this assignment context, and all existing uses are still fine. 'type' wouldn't be a keyword. To support the function call syntax lazily, we want to be able to distinguish between types and non-types in arguments. First, any literals would be eagerly calculated and not escaped: type T = f(1, "y", kw=False) # evaluated as T = f(1, "y", kw=False) Name expressions are lazy, to allow forward references: type T = f(C, kw=D) # T = f("C", kw="D") [if using PEP 563] If using delayed evaluation instead of stringification, we can create some "lazy type reference" objects instead of strings (this is kind of hand-wavy). Similarly, tuples, lists and dictionaries would be eager (but items can be lazy): type T = f(C, [("x", list[C])]) # T = f("C", [("x", "list[C]")]) This would resemble PEP 563 stringification, but it needs to be a little more flexible. The type statement syntax is general enough to support various existing use cases, including NewType, TypeVar, TypedDict, NamedTuple, and others, and it's also extensible to new forms. I'll write separate posts about my other ideas I mentioned earlier. Jukka
Just two random thoughts, not intended as an in-depth analysis of this proposal. Am 16.04.21 um 18:59 schrieb Jukka Lehtosalo:
However, what about type aliases:
A = () => int # Hmm?
An alternative for aliases is to rethink the rejected idea from PEP 613 and allow type aliases to be defined like this: A: TypeAlias[() => int] (I really wish we could use <> instead of [], though.)
type A = list[int]
As introducing "type" as a keyword could be problematic (although I might be mistaken), we could reuse "def": def A = list[int] - Sebastian
On Fri, Apr 16, 2021 at 6:24 PM Sebastian Rittau <srittau@rittau.biz> wrote:
Am 16.04.21 um 18:59 schrieb Jukka Lehtosalo:
However, what about type aliases:
A = () => int # Hmm?
An alternative for aliases is to rethink the rejected idea from PEP 613 and allow type aliases to be defined like this:
A: TypeAlias[() => int]
This has the problem that A has no value at runtime, so it can only be used in a type context, and we'd still need string literal escaping for cast("A", f), etc. It would also be a bit tricky to use for runtime introspection, and I don't want to regress things for users of pydantic, etc. -- though runtime introspection might work, since the target type is stored in __annotations__.
type A = list[int]
As introducing "type" as a keyword could be problematic (although I might be mistaken), we could reuse "def":
def A = list[int]
I *believe* that the new parser can handle "type" (i.e. context-sensitive keywords), but I'm not sure. "def" sounds like the best option from the existing real keywords. Jukka
This proposal also naturally extends to: type import mod and from mod type import X, Y These use cases don't need the custom syntax support, but with the same lazy evaluation they would form a natural replacement for guarding imports with `if TYPE_CHECKING:`, which would also be compatible with PEP 649 and runtime evaluation of annotations. Carl On Fri, Apr 16, 2021 at 11:00 AM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:
Many people seem to agree that Python typing still has some rough edges. Various PEPs have fixed warts (or proposed fixes), including PEPs 585, 563, and 649. Still, several issues remain:
1. Type annotation syntax is not always elegant (consider callable types). There's no good way to add custom type syntax without "polluting" non-typing use cases.
2. String literal escaping is ugly, and it's still needed from time to time.
3. Runtime introspection of types feels like a second-class citizen, even though it's pretty popular (pydantic, etc.).
Instead of trying to tackle these issues piecemeal, I've tried to come up with ideas to solve all of the above issues. They don't form a complete proposal, and I haven't worked out many details yet, but I'm hoping that this could be a discussion starter, and perhaps it can help us eventually design something that is workable.
Here I explain my first idea -- type assignment. Right now, we can't easily add dedicated syntax for callable types, for example, without either making this syntax available everywhere, or making it *un*available in type aliases and other common use cases, which don't use the annotation syntax.
Supporting new syntax in annotations seems manageable:
def doit(f: () => int) -> None: ... # Could be supported
However, what about type aliases:
A = () => int # Hmm?
If this works, nothing stops people from writing weird code like this:
d = {'x': (1 + y) => foo(1, 2)}
I'd rather have dedicated typing syntax reserved for type-related uses, mainly to avoid confusing Python users who don't use typing. If typing syntax is only available via a small number of special syntactic forms, it's easier to teach users to ignore it, if they don't care about it.
I'm proposing a new construct called "type assignment" where the RHS is lazily evaluated, and in the future it might support typing-only syntax. There would be two variants. First, we can define a type alias like this:
type A = list[int]
If we'd use the new syntax for callable types, we could write an alias like this:
type C = (int) => str # No problem
However, the following would be a syntax error, since it's not in a "type context":
C = (int) => str # Error, as it probably should be
Since the right hand side of type assignment contains types, we can avoid evaluating them during module import time, using either PEP 563 semantics (stringification) or something resembling PEP 649 (lazy evaluation). This means that we won't need to use explicit string literal escaping:
type A = B # OK! class B: ...
If the RHS is a function call, we'd call the function at runtime, but arguments would be (partially) stringified/lazy and they can contain type syntax (I'm glossing over some details):
type N = NewType(int) type T = TypeVar(bound=(int) => str) # Type-specific syntax is ok type D = typing.TypedDict({"x": C}) # C can be a forward reference! class C: ...
Note that the results here are usable in runtime contexts, since we call the function:
x = N(2) # Ok
The general syntax would be "type <id> = <name>(...)". We'd pass the stringified name as an implicit first argument for a cleaner syntax.
We can write the earlier examples today like this:
T = TypeVar("T", bound=Callable[[int], str]) N = NewType("N", int) D = typing.TypedDict("D", {"x": "C"}) # Need escaping here class C: ...
The new syntax uses 'type', which is widely used as a name. The new meaning would only apply in this assignment context, and all existing uses are still fine. 'type' wouldn't be a keyword.
To support the function call syntax lazily, we want to be able to distinguish between types and non-types in arguments.
First, any literals would be eagerly calculated and not escaped:
type T = f(1, "y", kw=False) # evaluated as T = f(1, "y", kw=False)
Name expressions are lazy, to allow forward references:
type T = f(C, kw=D) # T = f("C", kw="D") [if using PEP 563]
If using delayed evaluation instead of stringification, we can create some "lazy type reference" objects instead of strings (this is kind of hand-wavy).
Similarly, tuples, lists and dictionaries would be eager (but items can be lazy):
type T = f(C, [("x", list[C])]) # T = f("C", [("x", "list[C]")])
This would resemble PEP 563 stringification, but it needs to be a little more flexible.
The type statement syntax is general enough to support various existing use cases, including NewType, TypeVar, TypedDict, NamedTuple, and others, and it's also extensible to new forms.
I'll write separate posts about my other ideas I mentioned earlier.
Jukka _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: carl@oddbird.net
Or perhaps type: import a from b import c def f() -> int: pass (You could even add an 'else:' clause. :-) On Fri, Apr 16, 2021 at 10:47 AM Carl Meyer <carl@oddbird.net> wrote:
This proposal also naturally extends to:
type import mod
and
from mod type import X, Y
These use cases don't need the custom syntax support, but with the same lazy evaluation they would form a natural replacement for guarding imports with `if TYPE_CHECKING:`, which would also be compatible with PEP 649 and runtime evaluation of annotations.
Carl
On Fri, Apr 16, 2021 at 11:00 AM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:
Many people seem to agree that Python typing still has some rough edges.
Various PEPs have fixed warts (or proposed fixes), including PEPs 585, 563, and 649. Still, several issues remain:
1. Type annotation syntax is not always elegant (consider callable
types). There's no good way to add custom type syntax without "polluting" non-typing use cases.
2. String literal escaping is ugly, and it's still needed from time to
time.
3. Runtime introspection of types feels like a second-class citizen,
even though it's pretty popular (pydantic, etc.).
Instead of trying to tackle these issues piecemeal, I've tried to come
up with ideas to solve all of the above issues. They don't form a complete proposal, and I haven't worked out many details yet, but I'm hoping that this could be a discussion starter, and perhaps it can help us eventually design something that is workable.
Here I explain my first idea -- type assignment. Right now, we can't
easily add dedicated syntax for callable types, for example, without either making this syntax available everywhere, or making it *un*available in type aliases and other common use cases, which don't use the annotation syntax.
Supporting new syntax in annotations seems manageable:
def doit(f: () => int) -> None: ... # Could be supported
However, what about type aliases:
A = () => int # Hmm?
If this works, nothing stops people from writing weird code like this:
d = {'x': (1 + y) => foo(1, 2)}
I'd rather have dedicated typing syntax reserved for type-related uses,
mainly to avoid confusing Python users who don't use typing. If typing syntax is only available via a small number of special syntactic forms, it's easier to teach users to ignore it, if they don't care about it.
I'm proposing a new construct called "type assignment" where the RHS is
lazily evaluated, and in the future it might support typing-only syntax. There would be two variants. First, we can define a type alias like this:
type A = list[int]
If we'd use the new syntax for callable types, we could write an alias
like this:
type C = (int) => str # No problem
However, the following would be a syntax error, since it's not in a
"type context":
C = (int) => str # Error, as it probably should be
Since the right hand side of type assignment contains types, we can
avoid evaluating them during module import time, using either PEP 563 semantics (stringification) or something resembling PEP 649 (lazy evaluation). This means that we won't need to use explicit string literal escaping:
type A = B # OK! class B: ...
If the RHS is a function call, we'd call the function at runtime, but
arguments would be (partially) stringified/lazy and they can contain type syntax (I'm glossing over some details):
type N = NewType(int) type T = TypeVar(bound=(int) => str) # Type-specific syntax is ok type D = typing.TypedDict({"x": C}) # C can be a forward reference! class C: ...
Note that the results here are usable in runtime contexts, since we call
the function:
x = N(2) # Ok
The general syntax would be "type <id> = <name>(...)". We'd pass the
stringified name as an implicit first argument for a cleaner syntax.
We can write the earlier examples today like this:
T = TypeVar("T", bound=Callable[[int], str]) N = NewType("N", int) D = typing.TypedDict("D", {"x": "C"}) # Need escaping here class C: ...
The new syntax uses 'type', which is widely used as a name. The new
meaning would only apply in this assignment context, and all existing uses are still fine. 'type' wouldn't be a keyword.
To support the function call syntax lazily, we want to be able to
distinguish between types and non-types in arguments.
First, any literals would be eagerly calculated and not escaped:
type T = f(1, "y", kw=False) # evaluated as T = f(1, "y", kw=False)
Name expressions are lazy, to allow forward references:
type T = f(C, kw=D) # T = f("C", kw="D") [if using PEP 563]
If using delayed evaluation instead of stringification, we can create
some "lazy type reference" objects instead of strings (this is kind of hand-wavy).
Similarly, tuples, lists and dictionaries would be eager (but items can
be lazy):
type T = f(C, [("x", list[C])]) # T = f("C", [("x", "list[C]")])
This would resemble PEP 563 stringification, but it needs to be a little
more flexible.
The type statement syntax is general enough to support various existing
use cases, including NewType, TypeVar, TypedDict, NamedTuple, and others, and it's also extensible to new forms.
I'll write separate posts about my other ideas I mentioned earlier.
Jukka _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: carl@oddbird.net
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Fri, Apr 16, 2021 at 5:14 PM Guido van Rossum <guido@python.org> wrote:
Or perhaps
type: import a from b import c def f() -> int: pass
Sure, that works too!
(You could even add an 'else:' clause. :-)
Since unlike `if TYPE_CHECKING:`, the semantics of `type:` would be that names defined in it are available at runtime (just lazily), I guess the semantics of the `else` would be that it runs eagerly and any names defined in it take precedence over those defined in `type:` at runtime? Carl
Eh, forget about adding ‘else’. On Sat, Apr 17, 2021 at 04:38 Carl Meyer <carl@oddbird.net> wrote:
On Fri, Apr 16, 2021 at 5:14 PM Guido van Rossum <guido@python.org> wrote:
Or perhaps
type: import a from b import c def f() -> int: pass
Sure, that works too!
(You could even add an 'else:' clause. :-)
Since unlike `if TYPE_CHECKING:`, the semantics of `type:` would be that names defined in it are available at runtime (just lazily), I guess the semantics of the `else` would be that it runs eagerly and any names defined in it take precedence over those defined in `type:` at runtime?
Carl
-- --Guido (mobile)
Agreed. A "type import" is the second feature that I think we'd need. The syntax I came up with wasn't as good as yours, though, so I'm not going to present it. Guarding imports with "if TYPE_CHECKING" is common, and right now it breaks all introspection and sometimes makes string quoting necessary, even with PEP 563 and PEP 649. Jukka On Fri, Apr 16, 2021 at 6:47 PM Carl Meyer <carl@oddbird.net> wrote:
This proposal also naturally extends to:
type import mod
and
from mod type import X, Y
These use cases don't need the custom syntax support, but with the same lazy evaluation they would form a natural replacement for guarding imports with `if TYPE_CHECKING:`, which would also be compatible with PEP 649 and runtime evaluation of annotations.
Carl
Hi Julia, On Fri, Apr 16, 2021 at 11:00 AM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:
If the RHS is a function call, we'd call the function at runtime, but arguments would be (partially) stringified/lazy and they can contain type syntax (I'm glossing over some details):
type N = NewType(int) type T = TypeVar(bound=(int) => str) # Type-specific syntax is ok type D = typing.TypedDict({"x": C}) # C can be a forward reference! class C: ...
Note that the results here are usable in runtime contexts, since we call the function:
x = N(2) # Ok
My main question about the details of your proposal is this piece. Partial laziness adds a lot of complexity both to implementation and simply to understanding the semantics; is it really necessary? Might it not be acceptable to say that `x = N(2)` can only occur after any forward references involved in the definition of `N` have been defined? (In the common cases I think it would occur inside functions; the ordering is only really an issue if it occurs at top level.) And if so, could we avoid the complexities of partial laziness and instead just always wrap the entire RHS in lazy evaluation? The other issue is implicitly passing the stringified name as first arg. Personally I would rather handle the lazy evaluation in a generalized way as something like “module level descriptors”, in which case the name issue could be handled via the existing `__set_name__` protocol instead. Overall I like the direction of the proposal, and agree that a more holistic solution to handling of type syntax at runtime is needed. Carl
On Sat, Apr 17, 2021 at 12:29 PM Carl Meyer <carl@oddbird.net> wrote:
On Fri, Apr 16, 2021 at 11:00 AM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:
Note that the results here are usable in runtime contexts, since we call the function:
x = N(2) # Ok
My main question about the details of your proposal is this piece. Partial laziness adds a lot of complexity both to implementation and simply to understanding the semantics; is it really necessary? Might it not be acceptable to say that `x = N(2)` can only occur after any forward references involved in the definition of `N` have been defined? (In the common cases I think it would occur inside functions; the ordering is only really an issue if it occurs at top level.) And if so, could we avoid the complexities of partial laziness and instead just always wrap the entire RHS in lazy evaluation?
The partial laziness seems helpful to avoid explicit escaping in use cases like this: type T = TypeVar(bound=D) class C(Generic[T]): ... # If we evaluate T fully, we get NameError about D class D: ... Code like this seems somewhat common. I could find over 100 forward references in type variable bounds in a large internal codebase. Not having partial laziness could result in getting a NameError potentially far away from the source, which can be confusing. Changing the order of definitions will sometimes help, but it's not always practical. It can create a lot of needless code churn, especially if we are talking about changing the order of big classes. Some forward references are caused by import cycles, and avoiding import cycles is often not possible without major compromises. If we'd limit the use of forward references, users might prefer the old way as more flexible. There are other possible tricky situations that involve base classes. Example: type NT = NamedTuple(x=D) class class C(NT): ... class D: ... Occasionally type objects are also stored in some module-level data structures. Hypothetical example: type X = NamedTuple(...) type Y = NamedTuple(...) dd = { 'key1': X, 'key2': Y, ... } The other issue is implicitly passing the stringified name as first arg.
Personally I would rather handle the lazy evaluation in a generalized way as something like “module level descriptors”, in which case the name issue could be handled via the existing `__set_name__` protocol instead.
I don't much like the magic first argument, to be honest. Something like an implicit call to '__set_name__' sounds better to me. I'd just like to be able to drop the explicit string literal name argument somehow, since it looks quite awkward -- especially if the type name is long. Jukka
On Mon, Apr 19, 2021 at 5:49 AM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:
The partial laziness seems helpful to avoid explicit escaping in use cases like this:
type T = TypeVar(bound=D) class C(Generic[T]): ... # If we evaluate T fully, we get NameError about D class D: ...
Code like this seems somewhat common. I could find over 100 forward references in type variable bounds in a large internal codebase. Not having partial laziness could result in getting a NameError potentially far away from the source, which can be confusing.
Changing the order of definitions will sometimes help, but it's not always practical. It can create a lot of needless code churn, especially if we are talking about changing the order of big classes. Some forward references are caused by import cycles, and avoiding import cycles is often not possible without major compromises. If we'd limit the use of forward references, users might prefer the old way as more flexible.
I think you're right. I've not been able to think of a better option than some form of partial laziness. I hope it could be kept as simple and consistent as possible.
I'd just like to be able to drop the explicit string literal name argument somehow, since it looks quite awkward -- especially if the type name is long.
Agreed. Carl
I wonder if it would be useful to let this syntax be used for overloads as well. The current overload syntax is quite verbose, but could perhaps look like this: type def a(arg: int) -> int: ... type def a(arg: str) -> str: ... type def a(arg: object) -> NoReturn: ... def a(arg: Union[int, str, object]) -> Union[int, str]: if not isinstance(arg, (int, str)): raise TypeError return arg
I'm supportive of type assignment syntax. It fills one of the final places where postponed evaluation of annotations wasn't possible.
The general syntax would be "type <id> = <name>(...)". We'd pass the stringified name as an implicit first argument for a cleaner syntax.
I will admit the special treatment of the first <id> parameter to the <name>(...) function gives me some pause, but no non-special treatment strategy comes to mind.
I'll write separate posts about my other ideas I mentioned earlier.
Looking forward to hearing about your other ideas as well. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy On 4/16/21 9:59 AM, Jukka Lehtosalo wrote:
Many people seem to agree that Python typing still has some rough edges. Various PEPs have fixed warts (or proposed fixes), including PEPs 585, 563, and 649. Still, several issues remain:
1. Type annotation syntax is not always elegant (consider callable types). There's no good way to add custom type syntax without "polluting" non-typing use cases.
2. String literal escaping is ugly, and it's still needed from time to time.
3. Runtime introspection of types feels like a second-class citizen, even though it's pretty popular (pydantic, etc.).
Instead of trying to tackle these issues piecemeal, I've tried to come up with ideas to solve all of the above issues. They don't form a complete proposal, and I haven't worked out many details yet, but I'm hoping that this could be a discussion starter, and perhaps it can help us eventually design something that is workable.
Here I explain my first idea -- type assignment. Right now, we can't easily add dedicated syntax for callable types, for example, without either making this syntax available everywhere, or making it *un*available in type aliases and other common use cases, which don't use the annotation syntax.
Supporting new syntax in annotations seems manageable:
def doit(f: () => int) -> None: ... # Could be supported
However, what about type aliases:
A = () => int # Hmm?
If this works, nothing stops people from writing weird code like this:
d = {'x': (1 + y) => foo(1, 2)}
I'd rather have dedicated typing syntax reserved for type-related uses, mainly to avoid confusing Python users who don't use typing. If typing syntax is only available via a small number of special syntactic forms, it's easier to teach users to ignore it, if they don't care about it.
I'm proposing a new construct called "type assignment" where the RHS is lazily evaluated, and in the future it might support typing-only syntax. There would be two variants. First, we can define a type alias like this:
type A = list[int]
If we'd use the new syntax for callable types, we could write an alias like this:
type C = (int) => str # No problem
However, the following would be a syntax error, since it's not in a "type context":
C = (int) => str # Error, as it probably should be
Since the right hand side of type assignment contains types, we can avoid evaluating them during module import time, using either PEP 563 semantics (stringification) or something resembling PEP 649 (lazy evaluation). This means that we won't need to use explicit string literal escaping:
type A = B # OK! class B: ...
If the RHS is a function call, we'd call the function at runtime, but arguments would be (partially) stringified/lazy and they can contain type syntax (I'm glossing over some details):
type N = NewType(int) type T = TypeVar(bound=(int) => str) # Type-specific syntax is ok type D = typing.TypedDict({"x": C}) # C can be a forward reference! class C: ...
Note that the results here are usable in runtime contexts, since we call the function:
x = N(2) # Ok
The general syntax would be "type <id> = <name>(...)". We'd pass the stringified name as an implicit first argument for a cleaner syntax.
We can write the earlier examples today like this:
T = TypeVar("T", bound=Callable[[int], str]) N = NewType("N", int) D = typing.TypedDict("D", {"x": "C"}) # Need escaping here class C: ...
The new syntax uses 'type', which is widely used as a name. The new meaning would only apply in this assignment context, and all existing uses are still fine. 'type' wouldn't be a keyword.
To support the function call syntax lazily, we want to be able to distinguish between types and non-types in arguments.
First, any literals would be eagerly calculated and not escaped:
type T = f(1, "y", kw=False) # evaluated as T = f(1, "y", kw=False)
Name expressions are lazy, to allow forward references:
type T = f(C, kw=D) # T = f("C", kw="D") [if using PEP 563]
If using delayed evaluation instead of stringification, we can create some "lazy type reference" objects instead of strings (this is kind of hand-wavy).
Similarly, tuples, lists and dictionaries would be eager (but items can be lazy):
type T = f(C, [("x", list[C])]) # T = f("C", [("x", "list[C]")])
This would resemble PEP 563 stringification, but it needs to be a little more flexible.
The type statement syntax is general enough to support various existing use cases, including NewType, TypeVar, TypedDict, NamedTuple, and others, and it's also extensible to new forms.
I'll write separate posts about my other ideas I mentioned earlier.
Jukka
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: davidfstr@gmail.com
I like the idea of having special syntax for types definitions. However, I'm curious how this would interact with the existing (type-free) way of defining types? Do you see type assignment eventually becoming the one and only way? Sergei On Fri, Apr 16, 2021 at 6:00 PM Jukka Lehtosalo <jlehtosalo@gmail.com> wrote:
Many people seem to agree that Python typing still has some rough edges. Various PEPs have fixed warts (or proposed fixes), including PEPs 585, 563, and 649. Still, several issues remain:
1. Type annotation syntax is not always elegant (consider callable types). There's no good way to add custom type syntax without "polluting" non-typing use cases.
2. String literal escaping is ugly, and it's still needed from time to time.
3. Runtime introspection of types feels like a second-class citizen, even though it's pretty popular (pydantic, etc.).
Instead of trying to tackle these issues piecemeal, I've tried to come up with ideas to solve all of the above issues. They don't form a complete proposal, and I haven't worked out many details yet, but I'm hoping that this could be a discussion starter, and perhaps it can help us eventually design something that is workable.
Here I explain my first idea -- type assignment. Right now, we can't easily add dedicated syntax for callable types, for example, without either making this syntax available everywhere, or making it *un*available in type aliases and other common use cases, which don't use the annotation syntax.
Supporting new syntax in annotations seems manageable:
def doit(f: () => int) -> None: ... # Could be supported
However, what about type aliases:
A = () => int # Hmm?
If this works, nothing stops people from writing weird code like this:
d = {'x': (1 + y) => foo(1, 2)}
I'd rather have dedicated typing syntax reserved for type-related uses, mainly to avoid confusing Python users who don't use typing. If typing syntax is only available via a small number of special syntactic forms, it's easier to teach users to ignore it, if they don't care about it.
I'm proposing a new construct called "type assignment" where the RHS is lazily evaluated, and in the future it might support typing-only syntax. There would be two variants. First, we can define a type alias like this:
type A = list[int]
If we'd use the new syntax for callable types, we could write an alias like this:
type C = (int) => str # No problem
However, the following would be a syntax error, since it's not in a "type context":
C = (int) => str # Error, as it probably should be
Since the right hand side of type assignment contains types, we can avoid evaluating them during module import time, using either PEP 563 semantics (stringification) or something resembling PEP 649 (lazy evaluation). This means that we won't need to use explicit string literal escaping:
type A = B # OK! class B: ...
If the RHS is a function call, we'd call the function at runtime, but arguments would be (partially) stringified/lazy and they can contain type syntax (I'm glossing over some details):
type N = NewType(int) type T = TypeVar(bound=(int) => str) # Type-specific syntax is ok type D = typing.TypedDict({"x": C}) # C can be a forward reference! class C: ...
Note that the results here are usable in runtime contexts, since we call the function:
x = N(2) # Ok
The general syntax would be "type <id> = <name>(...)". We'd pass the stringified name as an implicit first argument for a cleaner syntax.
We can write the earlier examples today like this:
T = TypeVar("T", bound=Callable[[int], str]) N = NewType("N", int) D = typing.TypedDict("D", {"x": "C"}) # Need escaping here class C: ...
The new syntax uses 'type', which is widely used as a name. The new meaning would only apply in this assignment context, and all existing uses are still fine. 'type' wouldn't be a keyword.
To support the function call syntax lazily, we want to be able to distinguish between types and non-types in arguments.
First, any literals would be eagerly calculated and not escaped:
type T = f(1, "y", kw=False) # evaluated as T = f(1, "y", kw=False)
Name expressions are lazy, to allow forward references:
type T = f(C, kw=D) # T = f("C", kw="D") [if using PEP 563]
If using delayed evaluation instead of stringification, we can create some "lazy type reference" objects instead of strings (this is kind of hand-wavy).
Similarly, tuples, lists and dictionaries would be eager (but items can be lazy):
type T = f(C, [("x", list[C])]) # T = f("C", [("x", "list[C]")])
This would resemble PEP 563 stringification, but it needs to be a little more flexible.
The type statement syntax is general enough to support various existing use cases, including NewType, TypeVar, TypedDict, NamedTuple, and others, and it's also extensible to new forms.
I'll write separate posts about my other ideas I mentioned earlier.
Jukka _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: sergei.a.lebedev@gmail.com
On Thu, Apr 29, 2021 at 8:22 PM Sergei Lebedev <sergei.a.lebedev@gmail.com> wrote:
I like the idea of having special syntax for types definitions. However, I'm curious how this would interact with the existing (type-free) way of defining types? Do you see type assignment eventually becoming the one and only way?
I'd expect that new code (that doesn't need to support older Python versions) would have little reason not to use the new syntax. It also seems feasible to write migration tools to translate existing code to use the new syntax. We wouldn't be able to remove the current type syntax any time soon, since it's used all over the place. I think that we can continue supporting the old syntax for legacy code indefinitely, since maintaining support for the old syntax seems easy enough. Jukka
participants (7)
-
Anton Agestam -
Carl Meyer -
David Foster -
Guido van Rossum -
Jukka Lehtosalo -
Sebastian Rittau -
Sergei Lebedev