A more generic solution to the @dataclass problem?
When I first stubbed my toe on dataclass typing the code I tried was this: ``` def mydataclass(...): ... return dataclasses.dataclass(...) mydataclass = typing.cast('dataclasses.dataclass', mydataclass) ``` I see now that this was wrong and naive, but perhaps you can see my intent there. I was trying to hint the type system that it should treat mydataclass exactly as it treats `@dataclass`. Do any of you see value in making such a thing work? It would solve a address a good deal more problems than PEP-681, in that it could help with custom properties, custom Enum, and the list goes on. More concretely, I propose a `typing.cast_by_value` (naming is hard) that will have the semantics of replacing itself with its first argument, as far as typechecking is concerned. This adds missing and needed functionality, because it's currently impossible to create anything other than `__builtins__.property` that is treated exactly like `__builtins__.property` by typechecking, especially in a standards-based way that will reliably work across type checking implementations. If my `property` example is a bad one, please use the strongest of these examples: `enum.Enum`, `dataclasses.dataclass`, `namedtuple`, `NamedTuple`.
Comments inline: On Fri, 2022-08-05 at 00:19 +0000, Buck Evan wrote:
When I first stubbed my toe on dataclass typing the code I tried was this:
``` def mydataclass(...): ... return dataclasses.dataclass(...)
mydataclass = typing.cast('dataclasses.dataclass', mydataclass) ```
I see now that this was wrong and naive, but perhaps you can see my intent there. I was trying to hint the type system that it should treat mydataclass exactly as it treats `@dataclass`.
You can in fact define your own `mydataclass` function that wraps the `dataclass` decorator function, and then decorate your own classes with `@mydataclass`. This is probably not your intent here though. In your example, I also do not understand why you're trying to cast your `mydataclass` function as the `dataclass` decorator; `dataclass` is not a type, it's a function.
Do any of you see value in making such a thing work? It would solve a address a good deal more problems than PEP-681, in that it could help with custom properties, custom Enum, and the list goes on.
I don't yet precisely understand what you're trying to make work. Are you trying to provide a hinting system for (static) type checkers that says, "pretend my X here is in fact a Y"?
More concretely, I propose a `typing.cast_by_value` (naming is hard) that will have the semantics of replacing itself with its first argument, as far as typechecking is concerned. This adds missing and needed functionality, because it's currently impossible to create anything other than `__builtins__.property` that is treated exactly like `__builtins__.property` by typechecking, especially in a standards-based way that will reliably work across type checking implementations. If my `property` example is a bad one, please use the strongest of these examples: `enum.Enum`, `dataclasses.dataclass`, `namedtuple`, `NamedTuple`.
Paul
There's a better and more flexible way to accomplish this kind of lie already: ``` if typing.TYPE_CHECKING: mydataclass = dataclasses.dataclass else: def mydataclass(...): ... ``` On Thu, 4 Aug 2022 at 19:11, Paul Bryan <pbryan@anode.ca> wrote:
Comments inline:
On Fri, 2022-08-05 at 00:19 +0000, Buck Evan wrote:
When I first stubbed my toe on dataclass typing the code I tried was this:
``` def mydataclass(...): ... return dataclasses.dataclass(...)
mydataclass = typing.cast('dataclasses.dataclass', mydataclass) ```
I see now that this was wrong and naive, but perhaps you can see my intent there. I was trying to hint the type system that it should treat mydataclass exactly as it treats `@dataclass`.
You can in fact define your own `mydataclass` function that wraps the `dataclass` decorator function, and then decorate your own classes with `@mydataclass <@mydataclass>`. This is probably not your intent here though. In your example, I also do not understand why you're trying to cast your `mydataclass` function as the `dataclass` decorator; `dataclass` is not a type, it's a function.
Do any of you see value in making such a thing work? It would solve a address a good deal more problems than PEP-681, in that it could help with custom properties, custom Enum, and the list goes on.
I don't yet precisely understand what you're trying to make work. Are you trying to provide a hinting system for (static) type checkers that says, "pretend my X here is in fact a Y"?
More concretely, I propose a `typing.cast_by_value` (naming is hard) that will have the semantics of replacing itself with its first argument, as far as typechecking is concerned. This adds missing and needed functionality, because it's currently impossible to create anything other than `__builtins__.property` that is treated exactly like `__builtins__.property` by typechecking, especially in a standards-based way that will reliably work across type checking implementations. If my `property` example is a bad one, please use the strongest of these examples: `enum.Enum`, `dataclasses.dataclass`, `namedtuple`, `NamedTuple`.
Paul
_______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: hauntsaninja@gmail.com
Paul Bryan wrote:
Comments inline: On Fri, 2022-08-05 at 00:19 +0000, Buck Evan wrote:
When I first stubbed my toe on dataclass typing the code I tried was this: def mydataclass(...): ... return dataclasses.dataclass(...)
mydataclass = typing.cast('dataclasses.dataclass', mydataclass)
I see now that this was wrong and naive, but perhaps you can see my intent there. I was trying to hint the type system that it should treat mydataclass exactly as it treats `@dataclass`. You can in fact define your own `mydataclass` function that wraps the `dataclass` decorator function, and then decorate your own classes with `@mydataclass`.
Yes, this is possible in python, but not in typed python. All type checkers will fail to understand what's happening. `demo.py`: ```python import dataclasses def mydataclass(): return dataclasses.dataclass() @mydataclass class C: x: int C(3) ``` ``` ```sh $ $ python demo.py C(x=3) $ mypy demo.py demo.py:9: error: Too many arguments for "C" $ pyright demo.py /home/bukzor/demo/demo.py:9:8 - error: Expected no arguments to "C" constructor (reportGeneralTypeIssues) $ pytype demo.py Success: no errors found $ pyre check demo.py:6:0 Uninitialized attribute [13]: Attribute `x` is declared in class `C` to have type `int` but is never initialized. demo.py:9:7 Too many arguments [19]: Call `object.__init__` expects 0 positional arguments, 1 was provided. ``` (Notably pytype gets this right, but only if the definition and use are in the same module. Otherwise, everything becomes typed to Any.) To be clear, I'm *not* saying that this in particular is an important use case. I'm only using it as one example of the quite generic difficulty of imbuing user-defined objects with the semantics found in blessed stdlib objects.
`dataclass` is not a type, it's a function.
I agree, and perhaps that's the crux of my entire complaint. If it were a type, then I could use type annotations to denote the correct semantics of this code. Because it's not a type, it's not possible for any type annotation to refer to the semantics attached to the *value* named `dataclasses.dataclass`.
Shantanu Jain wrote:
There's a better and more flexible way to accomplish this kind of lie already: ``` if typing.TYPE_CHECKING: mydataclass = dataclasses.dataclass else: def mydataclass(...): ... ```
Would PEP681 exist if that were true? I like this solution, but because the semantics of `dataclass` is not bound to a type (it's bound to the value), typecheckers can't (generally) understand it, even without any conditional: `demo.py`: ```python import dataclasses mydataclass = dataclasses.dataclass @mydataclass() class C: x: int c: C = C(3) print(c) ``` ```sh $ python demo.py C(x=3) $ mypy demo.py demo.py:8: error: Too many arguments for "C" Found 1 error in 1 file (checked 1 source file) $ pyre check ƛ Found 2 type errors! demo.py:5:0 Uninitialized attribute [13]: Attribute `x` is declared in class `C` to have type `int` but is never initialized. demo.py:8:7 Too many arguments [19]: Call `object.__init__` expects 0 positional arguments, 1 was provided. $ pytype demo.py Success: no errors found $ pyright demo.py 0 errors, 0 warnings, 0 informations ``` If you move the assignment off to another module though, none of the typecheckers understand it*. I believe it's because .pyi, as the standard inter-module type representation, can't represent such value-level shenanigans. *(pytype "succeeds" because the relevant types are inferred as Any) --- To be clear, I'm not arguing that this is a *big* problem, but it is a problem that I've personally hit time and time again, and it commonly prevents my projects from being well-typed. My goal in this discussion is to find some kind of community agreement on A) whether a problem exists and B) how to describe the problem (if any). I have several more pitches for how to improve the situation, but I'm refraining because I don't want to further muddy the discussion.
Shantanu Jain wrote:
There's a better and more flexible way to accomplish this kind of lie already: ``` if typing.TYPE_CHECKING: mydataclass = dataclasses.dataclass else: def mydataclass(...): ... ```
Would PEP681 exist if that were true?
Shantanu's solution works for simple cases where something works *exactly*
El vie, 5 ago 2022 a las 10:13, Buck Evan (<buck.2019@gmail.com>) escribió: like @dataclass. PEP 681 extends to more cases where a library provides broadly dataclass-like semantics, but e.g. using a metaclass instead of a decorator.
because it's currently impossible to create anything other than `__builtins__.property` that is treated exactly like `__builtins__.property` by typechecking ... these examples: `enum.Enum`, `dataclasses.dataclass`, `namedtuple`, `NamedTuple`
This version should work with mypy and across modules: ``` from typing import TYPE_CHECKING __all__ = ["mdc"] if TYPE_CHECKING: from dataclasses import dataclass as mdc else: def mdc(*a, **k): ... @mdc class X: x: int X("error") ``` On Fri, 5 Aug 2022 at 10:17, Jelle Zijlstra <jelle.zijlstra@gmail.com> wrote:
El vie, 5 ago 2022 a las 10:13, Buck Evan (<buck.2019@gmail.com>) escribió:
Shantanu Jain wrote:
There's a better and more flexible way to accomplish this kind of lie already: ``` if typing.TYPE_CHECKING: mydataclass = dataclasses.dataclass else: def mydataclass(...): ... ```
Would PEP681 exist if that were true?
Shantanu's solution works for simple cases where something works *exactly* like @dataclass. PEP 681 extends to more cases where a library provides broadly dataclass-like semantics, but e.g. using a metaclass instead of a decorator.
Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: hauntsaninja@gmail.com
participants (4)
-
Buck Evan -
Jelle Zijlstra -
Paul Bryan -
Shantanu Jain