That's a potential workaround as `Required` is pretty self-explanatory indeed. Though, you would expect the same to work in reverse: class MyThing(TypedDict, total=True): req1: int opt1: NotRequired[str] # or whatever we call it req2: float Having support for just the one way around would feel insufficient... 1) How about: Droppable Potential Open 2) I'm actually wondering where the "optionality" here should be indicated. In typescript, this is marked next to the variable and not the type: interface Data { opt?: string; } So should we instead: class MyThing(TypedDict): Optional[opt1]: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have null value That would maybe allow to re-use `Optional` as it would exist in different context (don't know how confusing that would be). But this probably doesn't fit the design and the parser?

Sorry, none of your proposals (Droppable, Potential, NotRequired, Open, or another metaclass flag) sound right to me. We don't have an inverse of Optional -- why would we need an inverse of Required? The only objection to Required I can think of is actually that it *sounds* like the inverse of Optional, which it isn't. Then again, Optional should have been named Nullable from the beginning, but it's too late to change it now. On Tue, Jan 5, 2021 at 12:36 AM Tuukka Mustonen wrote:

3) One more suggestion:

class MyThing(TypedDict, optional_types_may_be_missing=True): req1: int opt1: Optional[str]

This would allow `Optional` fields to be also missing. It would not work when you want a field, that can be missing, but if present, is not allowed to have `None` as value. I would expect that to be rare (but still a requirement for some).

Obviously a better name would be needed for the parameter. Maybe `allow_missing`, `missing`, `strict=False` etc. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

I don't disagree with you. But the fact is that the perfect word, "optional", has already been taken and a compromise would be needed (assuming there is a feature worth debating here).

We don't have an inverse of Optional -- why would we need an inverse of Required? The only objection to Required I can think of is actually that it sounds like the inverse of Optional, which it isn't.

Maybe it was confusing when I wrote "the other way around". If we consider: - Required = always there - Opposite of it = never there ("missing"?) - Optional = sometimes there I didn't mean the opposite/inverse, I meant "sometimes there". Anyway, I don't think it's a problem. Required/optional is common terminology, and pretty self-explanatory even to non-native English speakers. Theoretically, `Missing` would be needed if you wanted to declare a Typescript-like interface: class MyThing(TypedDict): [key: str]: int denied: Missing ...which would allow arbitrary key pairs but not field with `denied` as key. But, this is highly theoretical, I don't think there's use case. So yes, I agree that inverse of "required" is not needed. Why I am after something else than your earlier proposal of: class MyThing(TypedDict, total=False): req1: Required[int] opt1: str req2: Required[float] ...is that people _expect_ it to work the other way around. It feels more natural if the stuff you define is expected to exist, and you then mark up what differs from that. That's how ~everything else already works and I would say that's how our brain are already tuned.

opt1: ~str

I think this would read as "inverse of string" (strictly) or "not string" (in a more loose sense), rather than as "perhaps not there"?

On Thu, Jan 7, 2021 at 11:35 PM David Foster wrote:

A1. On 1/6/21 3:54 AM, Tuukka Mustonen wrote:

...

Why I am after something else than your earlier proposal of:

class MyThing(TypedDict, total=False): req1: Required[int] opt1: str req2: Required[float]

...is that people _expect_ it to work the other way around. It feels more natural if the stuff you define is expected to exist, and you then mark up what differs from that. That's how ~everything else already works and I would say that's how our brain are already tuned.

Agreed. I would think "required" would be assumed to be the default, and "not-required" would be the special case that needs marking.

But "required" *is* the default -- you have to specify `total=False` to make fields non-required.

B1. On 1/6/21 3:54 AM, Tuukka Mustonen wrote:

...

opt1: NotRequired[str] # or whatever we call it

I actually like NotRequired.

It's pretty ugly (though there's a precedent, `NoReturn` for functions that don't return). It's also long and difficult to type due to the camelcase. If there were a better word I'd be okay with doing it this way around, but NotRequired just doesn't cut it for me. Hence my proposal.

However I can also see some objections: * It feels a bit odd to bless a term that starts with "not", in negative form rather than some positive form. * A one-word term seems preferable over a two-word term like NotRequired.

What I said. :-)

B2. On 1/5/21 12:03 AM, Tuukka Mustonen wrote:

...

Droppable

Mmm. The Rustacean in me thinks this means "deallocatable" which isn't correct here. -1

...

Potential

Mmm. Feels too vague. -1

...

Open

Mmm. Feels like you could apply "open" to an entire class/structure, but not to an individual member. -1

B3. I will also contribute a few bikeshed colors:

* OptionalKey * MissingOk

...

So should we instead:

class MyThing(TypedDict): Optional[opt1]: str # may not exist, but if exists, value is string opt2: Optional[str] # always exists, but may have null value

That would maybe allow to re-use `Optional` as it would exist in different context (don't know how confusing that would be). But this

C1. On 1/5/21 12:03 AM, Tuukka Mustonen wrote: probably doesn't fit the design and the parser?

Interesting. I like how this looks, and it doesn't require us to introduce a new (key-only) synonym for Optional.

But as you say it's not clear to me how it would be parsed exactly.

Also, we already have precedent from Final[] and Annotated[] that suggests that properties of a variable (as opposed to properties of the value shape) still be put into the type hint that wraps the other component of the type that contains the value shape.

C2. If we *do* wrap the right-hand-side rather than the left-hand-side, any spelling we have for "the key for this value can be missing" must also support interacting with the existing Optional spelling without looking too strange. For example:

class MyThing(TypedDict): opt: OptionalKey[Optional[str]]

class MyThing(TypedDict): opt: MissingOk[Optional[str]]

class MyThing(TypedDict): opt: NotRequired[Optional[str]]

The last example above in particular looks pretty strange to me.

Let's just not put funny syntax before the colon. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Sorry if I add to bikeshedding, but would it make sense to have `Missing` special value that's used by Union? i.e. you'd write: class Person(TypeDict): name: str date_birth: date date_death: date|Missing Even from an internal typechecker standpoint I think it would make formal sense to say that {"name": "Alex"}["not_a_name"] has type Missing, and many rules on unions would work seamlessly and coherently. On Sat, 9 Jan 2021 at 16:49, Tuukka Mustonen wrote:

...

But "required" is the default -- you have to specify total=False to make fields non-required.

Yeah, required is the default. But the convenience syntax that we're discussing here wouldn't work with that. Having to specify `total=False` and then `Required` for required fields is not intuitive. It would be a good addition and a workaround, but it doesn't fulfill the need here.

...

It's pretty ugly (though there's a precedent, NoReturn for functions that don't return). It's also long and difficult to type due to the camelcase.

I also think most proposals look ugly. However, I'm not sure if it makes sense to reject a feature just because perfect wording cannot be found? People who can't live with the name could create an alias anyway, like:

O = NotRequired

That's probably what I would do - not to avoid ugliness but simply to make the syntax less verbose.

Sure, I could do the same with `R = Required` but it doesn't fix the problem of having to think about it "the other way around".

...

Let's just not put funny syntax before the colon.

Is it silly? If we consider typescript-like syntax again:

interface Data { opt?: str opt: str? }

The first one reads as "opt may be there?" while the second one reads as "type may be string?". Doing `NotRequired[str]` reads as "type may be string but doesn't have to be" or perhaps as "type is not required but may be string". _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: dfmoisset@gmail.com

Honestly I'm not keen on this type of solution. No matter what we do, unless we introduce new *syntax* (e.g. `age?: int`) we're already making a statement about the *variable* in the form of a statement about the *type*. We have precedents for that, but they are things like `Final[...]` and `ClassVar[...]`, and I think it would be better to continue along that line. We can make a simple rule that "Required" (for example) may only appear at the top level in a typed dict -- this seems simpler than a rule that says "one branch of a union can be a magic cookie, but only at the top level". In terms of perceived economy, regardless of whether we reuse an existing keyword or add a new one, the behavior for this particular case has to be learned and implemented separately, so I don't think we're saving much by avoiding a new keyword. (If we could reuse it would be a little different, but we clearly can't.) On Mon, Jan 11, 2021 at 10:40 AM Jelle Zijlstra wrote:

El lun, 11 ene 2021 a las 10:32, Daniel Moisset () escribió:

...

Sorry if I add to bikeshedding, but would it make sense to have `Missing` special value that's used by Union? i.e. you'd write:

class Person(TypeDict): name: str date_birth: date date_death: date|Missing

Even from an internal typechecker standpoint I think it would make formal sense to say that {"name": "Alex"}["not_a_name"] has type Missing, and many rules on unions would work seamlessly and coherently.

Perhaps `typing.NoReturn` could be used for this purpose too.

...

On Sat, 9 Jan 2021 at 16:49, Tuukka Mustonen wrote:

...

...

But "required" is the default -- you have to specify total=False to make fields non-required.

Yeah, required is the default. But the convenience syntax that we're discussing here wouldn't work with that. Having to specify `total=False` and then `Required` for required fields is not intuitive. It would be a good addition and a workaround, but it doesn't fulfill the need here.

...

It's pretty ugly (though there's a precedent, NoReturn for functions that don't return). It's also long and difficult to type due to the camelcase.

I also think most proposals look ugly. However, I'm not sure if it makes sense to reject a feature just because perfect wording cannot be found? People who can't live with the name could create an alias anyway, like:

O = NotRequired

That's probably what I would do - not to avoid ugliness but simply to make the syntax less verbose.

Sure, I could do the same with `R = Required` but it doesn't fix the problem of having to think about it "the other way around".

...

Let's just not put funny syntax before the colon.

Is it silly? If we consider typescript-like syntax again:

interface Data { opt?: str opt: str? }

The first one reads as "opt may be there?" while the second one reads as "type may be string?". Doing `NotRequired[str]` reads as "type may be string but doesn't have to be" or perhaps as "type is not required but may be string". _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: dfmoisset@gmail.com

_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com

_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

If we want the marker to go on the not-required keys, a possible singe-word positive bikeshed is Excludable class MyThing(TypedDict, total=True): req1: int opt1: Excludable[str] req2: float On Sat, 16 Jan 2021 at 19:48, David Foster wrote:

Catching up...

On 1/8/21 11:51 AM, Guido van Rossum wrote:> On Thu, Jan 7, 2021 at 11:35 PM David Foster

...

mailto:davidfstr@gmail.com> wrote:

A1. On 1/6/21 3:54 AM, Tuukka Mustonen wrote: > Why I am after something else than your earlier proposal of: > > class MyThing(TypedDict, total=False): > req1: Required[int] > opt1: str > req2: Required[float] > > ...is that people _expect_ it to work the other way around. It feels > more natural if the stuff you define is expected to exist, and you then > mark up what differs from that. That's how ~everything else already > works and I would say that's how our brain are already tuned.

Agreed. I would think "required" would be assumed to be the default, and "not-required" would be the special case that needs marking.

But "required" *is* the default -- you have to specify `total=False` to make fields non-required.

I think I've finally wrapped my head around this comment. In long-form:

* If you have a TypedDict with total=True (the default), then all keys are required. * If you have a TypedDict with total=False, then all keys become not-required by default. * Therefore introducing a Required[...] form is a backward-compatible way of marking certain keys of a total=False TypedDict as *required*, which currently isn't possible to do.

It bends my brain a bit: If I want to create an entirely new TypedDict with some keys required and some keys not-required then I must: (1) declare a total=False TypedDict and (2) explicitly mark the keys I want required with Required[...] and (3) remember that in this context all unmarked keys are NOT-required.

(This is backwards from TypeScript which essentially always declares TypedDicts as total=True but then only marks certain keys as NOT-required using the "name?: type" syntax.)

I suppose total=False TypedDicts bend my brain a bit anyway: It strikes me as odd that a single keyword at the top can flip the default required-ness of every key in the dictionary. However the total=False syntax is here now and I don't think it's going away, so we might as well work with it.

Long story short: I think Required[...] on a total=False TypedDict would be reasonable, given that the total=False syntax already exists. +0.5

Also, Required[...] satisfies: ✅ Single word. Easy to type. ✅ Is in positive form, rather than a "not X" or "no X" negative form. ✅ Does not "put funny syntax before the colon"

-- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: hauntsaninja@gmail.com

I don't think it's feasible to deprecate `Optional[x]`. It's too ubiquitous. However, I predict that it its use will fade over time in favor of `x | None` as specified in [PEP 604](https://www.python.org/dev/peps/pep-0604/).

I think one of the considerations that hasn't been brought up here is accessibility for people who only occasionally write Python or people who have to switch forth and back between languages (in our case between Hack/Flow and Python). From that perspective there's a strong case to be made for working towards syntax changes that other languages seem to agree on rather than introducing yet more clunky overhead. It also makes the argument in favor of `T | None` for optionals more difficult to make. Sure, it's short. But it's also difficult to explain to anyone outside of our community why we're using the value `None` to stand for the type of the value. It's impossible for people new to the language to reason about this from first principles. On Mon, Jan 18, 2021 at 10:35 AM layday via Typing-sig wrote:

One thing to note I suppose is that `name?` does not mean `T | null` in TS; it means something akin to `T | undefined`. `Required[T?]` might appear contradictory to someone coming from TS and I would personally prefer that the community converges on `T | None` - it's short and clear enough.

Best

D

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Monday, January 18, 2021 6:30 PM, David Foster wrote:

...

On 1/16/21 7:48 PM, David Foster wrote:

...

Long story short: I think Required[...] on a total=False TypedDict would be reasonable, given that the total=False syntax already exists. +0.5 Also, Required[...] satisfies: ✅ Single word. Easy to type. ✅ Is in positive form, rather than a "not X" or "no X" negative form. ✅ Does not "put funny syntax before the colon"

Let me play devil's advocate against myself for a moment here...

Let's say we use the form Required[...] only on total=False TypedDict definitions. You could encounter code like:

class MyTypedDict(TypedDict, total=False): foo: Required[str] bar: Required[Optional[int]] baz: Optional[float] qed: bool

Now to someone not intimately familiar with the differences between Required[...] and Optional[...], what kinds of questions or confusion could arise here?

- "It looks like `foo` and `bar` are required keys, and `baz` is an optional key, but what is the `qed` key at the end? Is it required? Is it optional?"

- "But wait a minute! The `bar` key is marked as both required AND as optional! What's going on here?"

A big problem you can see here is that it's very easy to conflate the concept of an "optional key" (no explicit spelling) with the concept of "nullable" (currently spelled Optional[...]).

It seems to me that any great spelling for "optional key" or "required key" (like Required[...]) is going to be significantly confusing so long as the current Optional[...] spelling (1) exists and (2) means "nullable" instead of "optional key".

So let me open a new can of worms: What if we deprecated the spelling Optional[...], replacing it with something like Nullable[...] instead?

My expectation is that PEP 645 ("Allow writing optional types as x?") will eventually phase out in-practice use of Optional[...] anyway, so deprecating the long-form spelling Optional[...] might not be so disruptive.

Now, if Optional[...] is deprecated it is less likely to show up in new user code, so my original code example would become:

class MyTypedDict(TypedDict, total=False): foo: Required[str] bar: Required[int?] baz: float? qed: bool

Here I think the user confusion in the original example goes away. "Required[...]" and "TYPE?" don't even look like the same kind of thing and therefore I expect would be hard to (mistakenly) see as opposites of each other.

(In case folks are wondering, after Optional[...] is deprecated and maybe even removed, I'm NOT presently advocating for eventually bringing it back with the new meaning of "optional key" rather than its current meaning of "nullable". There's probably a lot of content on the internet that already assigns the old meaning to Optional[...], so we can't change it's interpretation without potentially confusing a lot of people who are reading old help resources.)

-- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: layday@protonmail.com

_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: dkgispam@gmail.com

On 1/18/21 9:19 AM, Eric Traut wrote:

I don't think it's feasible to deprecate `Optional[x]`. It's too ubiquitous. However, I predict that it its use will fade over time in favor of `x | None` as specified in [PEP 604](https://www.python.org/dev/peps/pep-0604/).

On 1/18/21 12:55 PM, Guido van Rossum wrote:

The syntax x? is quite controversial and unlikely to be accepted anytime soon (and requires parser changes so invites much more scrutiny).

Points taken. I'll restate my last argument using this new information: Let's say we use the form Required[...] only on total=False TypedDict definitions. You could encounter code like: class MyTypedDict(TypedDict, total=False): foo: Required[str] bar: Required[int|None] baz: float|None qed: bool This code seems fairly straightforward to me. So long as we discourage in documentation the use of Optional[...] in the presence of Required[...] -- perhaps explicitly in favor of the `T|None` form -- it looks to me like Required[...] would be readable even in a more complex (real-world) case like this definition shows. Any major objections to the Required[...] syntax (in combination with total=False) being the solution to defining the optionality/requiredness of individual fields on TypedDicts? I'm feeling like the discussion and decisions so far are ripe to make into an actual PEP. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

On 1/24/21 8:58 PM, Guido van Rossum wrote:

I'm all for proceeding to the PEP stage.

Cool. I seem to be in a design mood this month, so I'd be happy to shepherd the drafting of a PEP for this feature.

Maybe you can even consider a PR for mypy?

Perhaps: we'll see if I have the bandwidth. I believe the extensions to mypy to support Required[...] wouldn't be that difficult, but I'm hoping to shepherd my work on TypeForm (both PEP and implementation) through first, and that I expect will be much more involved to implement. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

On 1/25/21 12:09 AM, Tuukka Mustonen wrote:

Just want to note that while this would be a good addition:

class MyTypedDict(TypedDict, total=False): foo: Required[str] bar: Required[int|None] baz: float|None qed: bool

...it's still "the wrong way around". This is what people usually want:

class MyTypedDict(TypedDict): foo: str bar: int|None baz: float|None|Missing qed: bool|Missing

I agree that it would be preferable to introduce a spelling for "optional key" rather than "required key", but there just doesn't seem to be a way to get a reasonable spelling: * We can't easily introduce any new single-word wrapper like Missing[...] because any good name that means "optional key" will necessarily be too close to Optional[...] and will be confusing when used in combination with Optional[...]. For example Optional[Missing[...]] is confusing. * So that leaves us with spelling the *opposite* of "optional key", which is a "required key". There's no name collision with any existing typing construct by just using Required[...], which is the obvious name for a "required key" wrapper.

Were there actually any comments on Daniel Moisset's suggestion of `| Missing`? Guido replied but maybe I don't understand the difference - if `Required[str]`, which defines the existence of the field (left side) attached to the type (right side), is considered acceptable approach, what's bad with `x|Missing`, which simply does the same?

The syntax `x|Missing` implies the introduction of a new constant (`Missing`) that's like `None`. Using such a constant as a magic flag is unusual. We typically use wrappers like Final[...], Annotated[...], etc instead. -- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy

On Wed, Jan 27, 2021 at 2:31 AM Tuukka Mustonen wrote:

Thanks for the clarification. So it's not that something couldn't be done, it's rather that too dramatic measures would need to be taken.

One more alternative that I suggested earlier, to which I failed to see any comments, was this:

class MyThing(TypedDict, optional_types_may_be_missing=True): req1: int opt1: Optional[str]

(Obviously, with some shorter name for `optional_types_may_be_missing` argument.)

It would make no distinction between "null" and "missing", but that's what one normally wants anyway. It wouldn't work to define a PATCH HTTP API endpoint, where sending out "null" (clear the value in DB) has different meaning from not sending the field at all (no change), but it would be sufficient for most cases. It's not perfect, but perhaps sufficient (like introducing the Required key).

This would add more confusion for users because it would mean that in *some* contexts the meaning of Optional[] is different than in other contexts. Can you please just get behind the Required[] proposal? -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

Just as an extra paragraph, note that these semantics make it about the values and not the variables as Guido said earlier On Wed, 27 Jan 2021 at 19:45, Daniel Moisset wrote:

I think that something in the proposal I sent was missed. The key of the proposal wasn't as much of "add a special value" (which it does), but mostly "use a Union[]" (I wrote it with the new | syntax, but I think this should be the same with explicit Union[Foo, Missing].

I stated this because the behaviour of "holes" in dictionaries already has Union semantics. Not just in the abstract sense of "there is more than one possible behaviour when evaluating mydict['mykey'] ", but in the general properties of Unions. For example consider this TypeDict declarations:

class A(TypedDict): x: int y: int|str z: int|Missing

class B(TypedDict): x: str z: int

value: A|B

Currently the type of value["x"] is int|str (the union of the types for x in both types). What happens for "holes" then?

value["z"] has type int|Missing. That is the Union[int|Missing, int] which again are the types on both sides (note that we use here that Unions can be flattened and int|int == int). value["y"] is a bit more trickier, but we can consider that there's an implicit "y: Missing" in B (for every name except "x" and "z" ) so the type of value["y"] is int|str|Missing. value["doesnotexist"] has type Missing|Missing, which due to union rules, is equivalent to Missing.

Adding some special form like "Required[int|str]" will need to define what does it mean to have Union[Required[T1], T2] and similar interactions, and in the end you will end up recreating the definition and algebraic rules for a Union, because what we have conceptually here is a Union. I know some people have expressed their concern that these standalone values are unusual (but not unheard of, we already have NoReturn), but for me this emerges from the type system.

In fact, I think we might get away defining Missing as an alias for NoReturn, because right now I see no much of a semantic difference . For example, any code after value["foo"] should be ignored in the same way as code after sys.exit() (a function with result type NoReturn). Anyway, this is an extra idea and if I am wrong this paragraph can be ignored without affecting the previous argument

Hope the example drives the idea more clearly

Best, D.

On Mon, 25 Jan 2021 at 08:10, Tuukka Mustonen wrote:

...

Just want to note that while this would be a good addition:

class MyTypedDict(TypedDict, total=False): foo: Required[str] bar: Required[int|None] baz: float|None qed: bool

...it's still "the wrong way around". This is what people usually want:

class MyTypedDict(TypedDict): foo: str bar: int|None baz: float|None|Missing qed: bool|Missing

Were there actually any comments on Daniel Moisset's suggestion of `| Missing`? Guido replied but maybe I don't understand the difference - if `Required[str]`, which defines the existence of the field (left side) attached to the type (right side), is considered acceptable approach, what's bad with `x|Missing`, which simply does the same? _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: dfmoisset@gmail.com

Adding a special type (`Missing`) to a union to indicate that a field is not required would be inconsistent with every other use of `Union`. It breaks the mental model users have for `Union`, and it would require type checkers to introduce a bunch of special-case logic in multiple places where unions are used. Please don't do this. The proposed `Required` annotation is consistent with existing annotations like `Final`, so it will be familiar to users and will work nicely with existing type checker implementations. I'll also note that `NoReturn` within a union makes no sense. The `NoReturn` type is semantically equivalent to an empty union. It should never appear within a union along with other types. Pyright contains special logic to remove it from a union if it is combined with other types. -Eric -- Eric Traut Contributor to Pyright and Pylance Microsoft Corp.

I think this is ready to be PEPped. Who wants to be the author? I can be sponsor. Or perhaps you want to start with a prototype implementation? Some additions to typeshed, typing_extensions and typing are also needed, I can help with some pointers. On Fri, Jan 29, 2021 at 04:05 Daniel Moisset wrote:

Ok, thanks for the detailed explanation. I stand corrected

On Thu, 28 Jan 2021, 03:44 Eric Traut, wrote:

...

Adding a special type (`Missing`) to a union to indicate that a field is not required would be inconsistent with every other use of `Union`. It breaks the mental model users have for `Union`, and it would require type checkers to introduce a bunch of special-case logic in multiple places where unions are used. Please don't do this.

The proposed `Required` annotation is consistent with existing annotations like `Final`, so it will be familiar to users and will work nicely with existing type checker implementations.

I'll also note that `NoReturn` within a union makes no sense. The `NoReturn` type is semantically equivalent to an empty union. It should never appear within a union along with other types. Pyright contains special logic to remove it from a union if it is combined with other types.

-Eric

-- Eric Traut Contributor to Pyright and Pylance Microsoft Corp. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: dfmoisset@gmail.com

_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido (mobile)

+1 On Sat, Jan 30, 2021 at 9:33 PM David Foster wrote:

On 1/29/21 7:32 AM, Guido van Rossum wrote:

...

I think this is ready to be PEPped. Who wants to be the author? I can be sponsor.

I'm game to draft this PEP. I feel like I've internalized the conversation fairly well. I'll start on that tonight.

I expect this feature will be a lot easier to implement than the TypeForm syntax I'm designing in a different thread. ;)

-- David Foster | Seattle, WA, USA Contributor to TypedDict support for mypy _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

The rationale is what the quoted paragraph says. Type checkers don’t track the *value* of expressions, so they don’t have the information needed for the check you desire. Some type checkers may go beyond this and keep track of values, but this is usually not as easy as it would seem from your example, and most code won’t benefit that much, so I don’t blame type checkers that don’t do this. —Guido On Fri, Feb 5, 2021 at 08:03 Vito De Tullio wrote:

Hi. Sorry if this is a little bit of a digression, but can I get the chance of a new PEP to ask for a more "strict" check of the optional/required keys of a TypedDict? Looking at the PEP589 I see:

...

...

...

Type checkers may allow reading an item using d['x'] even if the key 'x' is not required, instead of requiring the use of d.get('x') or an explicit 'x' in d check. The rationale is that tracking the existence of keys is difficult to implement in full generality, and that disallowing this could require many changes to existing code.

In fact, I see no error actually raised by current mypy on this snipped of code:

from typing import TypedDict

class _ARequired(TypedDict): required: int

class A(_ARequired, total=False): optional: int

A(required=123).get('required', 456) # no way the fallback value is used A(required=123)['optional'] # KeyError

I'm adding TypedDict in my project as a way to "describe" the response of some external API I consume, so I would really appreciate if I could be "warned" against "dead code" or, worse, risky one.

I'm not sure about the meaning of the rationale I quoted - the problem is in the user codebase checked? Or are internal difficulties of the type checker? In the first case - at least for me - I think that having stricter checks could help showing latent bugs.

FWIW I also think that "total=False" + "Required[...]" is the best syntax, given the backcompatibility with the actual semantic of TypedDict _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido (mobile)

You are tying two unrelated issues together. You're welcome to file bugs in the mypy tracker (or in the tracker of whatever checker you use) to ask for such checking as a feature. However, that feature does not require a PEP and this PEP should not be involved in requesting the feature. On Sun, Feb 7, 2021 at 8:37 AM Vito De Tullio wrote:

Hi.

Sorry for the misunderstanding. I understand that keep track of the value of each instance can be cumbersone (and it's not exactly in the scope of a *type* checker). Let me focus on the "type definition" of a TypedDict. Currently with TypedDict you can define all the 'required' or, with `total=False`, 'possible' keys a dict can have.

I can understand how can be difficult to deal with current "non total" TypedDict, and how the simple flag takes to a "too coarse" definition, exacerbated in "non total" TypedDict with a lot of keys that can take to a lot of false positives.

Still, now we are talking about giving a more "precise" definition of what keys are required, what are possible, and what are simply not present. How are the implementors suppose to use this additional information?

What I mean is: let's say that now we use the `Required` annotation:

from typing import TypedDict, Required

class C(TypedDict, total=False): required: Required[int] optional: int

def h(c: C, fallback: int) -> None: getitem_required = c['required'] getitem_optional = c['optional'] getitem_nonexistent = c['nonexistent']

get_required = c.get('required, fallback) get_optional = c.get('optional', fallback) get_existent = c.get('nonexistent', fallback)

I think it's a "waste" to not use the additional explicit annotation and mark as error, in addition to the `getitem_nonexistent` and `get_existent` examples, also the `getitem_optional` and the `get_required` rows.

Both the cases are code smell: in the first case is an hidden bug; the second one is basically dead code. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: guido@python.org

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...