Right way to dataclass -> dict conversion.
Example task: ``` from dataclasses import dataclass, asdict, fields from typing import List @dataclass class Point: x: int y: int @dataclass class Axes: title: str points: List[Point] def plot(title: str, points: List[Point]): print(title) for point in points: print(f'x={point.x}', f'y={point.y}') axes = Axes('example', [Point(i, i) for i in range(3)]) # Now I need to plot my axes. ``` Way 0: `plot(axes.title, axes.points)` Yes, it works great, but the topic about `dataclass2dict` conversion. Way 1: `plot(**asdict(axes))` Using `asdict` is what comes to mind first. But it is recursive, so it won't work. `asdict`, `astuple`... is an "bad naming" example, I think. Unexpected name for a recursive function. People don't read the documentation, and even when they do, they forget. As an example: https://stackoverflow.com/q/52229521/6146442 Viewed 7k times. Way 2: `plot(**vars(axes))` Everything seems to be good. But pseudo-fields which are `ClassVar` or `InitVar` will be returned. `asdict` use: `return tuple(f for f in fields.values() if f._field_type is _FIELD)` Way 3: Using undocumented `__dataclass_fields__` is look like a hack. Way 4: `plot(**{f.name: getattr(axes, f.name) for f in fields(axes)})` This will actually work. But I'm not sure about the elegance. My suggestions: 1. Add default `True` `recursive` flag to `asdict` and implement Way 4 internally. At the same time, IDE will remind people from stackoverflow that `asdict` is recursive. 2. Or add new not-recursive function to `dataclasses` module. Leave `asdict` frozen. 3. Do nothing. Way 4 is handsome enough. Describe the way in the documentation so that people don't get confused and don't use `vars` or `asdict`. I am ready to offer my assistance in implementation. Thank you for your attention and happy holidays!
Way 5: Add default `False` `iter` flag to `dataclass` decorator. ``` def __iter__(self) : return ((f.name, getattr(axes, f.name)) for f in fields(axes)) ``` And use: `plot(**dict(axes))`
I think you are getting a bit tangled up in the distinction between what I call "data" and "code": dicts are data dataclasses are code. Granted, in Python this line can be blurry, but I think it's helpful to think about it that way. Thinking of it this way, dataclasses.asdict() is converting code to data, and so it makes sense to convert it completely to data. In many contexts where you want a dict, a custom dataclass (or anything other custom class) as values wouldn't really make sense (imagine passing it into the json lib, for instance). All that being said, an option to do a "shallow" or "deep" conversion in asdict() makes sense to me. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Thu, Dec 31, 2020 at 11:51 AM Eric V. Smith <eric@trueblade.com> wrote:
I'd suggest you just write your own function to do this. I consider adding dataclasses.asdict and .astuple to be mistakes.
I agree there -- if you want something that can be used like this -- use a dict in the first place :-) Eric: what do you think about adding a "shallow" (or "deep") flag to dataclasses.asdict() that would then upack only the top-level dataclass? -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On 12/31/2020 3:59 PM, Eric V. Smith wrote:
Currently a deep copy is made: @dataclass class F: l: Optional[list] f: Optional["F"] mylist = [1, 2, 3] f = F(mylist, None) f1 = F(mylist, f)
f = F(mylist, None) f1 = F(mylist, f)
I think what we'd want is to just turn the top-level dataclass in to a dict, but then not recurse _and_ only make a shallow copy. Something like:
Since the 'f' member wouldn't be a copy (I hope there's agreement here), I'd think the 'l' member should also not be a copy. The question then is: what's a good name for my "mumble_mumble" argument? Does "deep=False" make sense? Or is it "recurse=False"? Or maybe it should just be a separate function. The intent of doing the shallow copy seems different than asdict() currently: asdict() is designed to turn everything to a dict, recursively. The target is json, but I'm not sure how well it fills that need. As I said, I don't think asdict() is a great example of API design, and I wish I'd left it out. Having some convoluted combination of not recursing but making a deep copy seems wrong to me. Eric
On Thu, Dec 31, 2020 at 4:29 PM Eric V. Smith <eric@trueblade.com> wrote:
Honestly, I'm a bit confused as to why arbitrary types were deep copied in the first place, but maybe that's just evidence that I'm not sure how useful this is in the first place.. or that it's not clear to me what the point was in the first place :-).
agreed.
The question then is: what's a good name for my "mumble_mumble" argument? Does "deep=False" make sense? Or is it "recurse=False"?
When this thread started, I didn't know that it was actually deep copying all members, and had suggested "deep" and "shallow" as analogous to copying. But as it really is doing copying, so i think "deep" and "shallow" are fine. Which leaves us with ``deep=False`` or ``shallow=True``. I usually prefer the default to be True, but in this case, I still like to thing of "deep" as the special case, so I'd go with: ``deep=False``. or ``deep_copy=False``.
well, yes.
The target is json, but I'm not sure how well it fills that need.
If you ask me, not very well, and there would have been no need to copy lists and the like for that purpose anyway. JSON-compatible Python is a specific thing -- you are either targeting that or not :-)
agreed on both points :-) One thing that *might* be useful is a recursing on dataclasses, but not copying anything else :-) -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Way 5: Add default `False` `iter` flag to `dataclass` decorator. ``` def __iter__(self) : return ((f.name, getattr(axes, f.name)) for f in fields(axes)) ``` And use: `plot(**dict(axes))`
I think you are getting a bit tangled up in the distinction between what I call "data" and "code": dicts are data dataclasses are code. Granted, in Python this line can be blurry, but I think it's helpful to think about it that way. Thinking of it this way, dataclasses.asdict() is converting code to data, and so it makes sense to convert it completely to data. In many contexts where you want a dict, a custom dataclass (or anything other custom class) as values wouldn't really make sense (imagine passing it into the json lib, for instance). All that being said, an option to do a "shallow" or "deep" conversion in asdict() makes sense to me. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Thu, Dec 31, 2020 at 11:51 AM Eric V. Smith <eric@trueblade.com> wrote:
I'd suggest you just write your own function to do this. I consider adding dataclasses.asdict and .astuple to be mistakes.
I agree there -- if you want something that can be used like this -- use a dict in the first place :-) Eric: what do you think about adding a "shallow" (or "deep") flag to dataclasses.asdict() that would then upack only the top-level dataclass? -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On 12/31/2020 3:59 PM, Eric V. Smith wrote:
Currently a deep copy is made: @dataclass class F: l: Optional[list] f: Optional["F"] mylist = [1, 2, 3] f = F(mylist, None) f1 = F(mylist, f)
f = F(mylist, None) f1 = F(mylist, f)
I think what we'd want is to just turn the top-level dataclass in to a dict, but then not recurse _and_ only make a shallow copy. Something like:
Since the 'f' member wouldn't be a copy (I hope there's agreement here), I'd think the 'l' member should also not be a copy. The question then is: what's a good name for my "mumble_mumble" argument? Does "deep=False" make sense? Or is it "recurse=False"? Or maybe it should just be a separate function. The intent of doing the shallow copy seems different than asdict() currently: asdict() is designed to turn everything to a dict, recursively. The target is json, but I'm not sure how well it fills that need. As I said, I don't think asdict() is a great example of API design, and I wish I'd left it out. Having some convoluted combination of not recursing but making a deep copy seems wrong to me. Eric
On Thu, Dec 31, 2020 at 4:29 PM Eric V. Smith <eric@trueblade.com> wrote:
Honestly, I'm a bit confused as to why arbitrary types were deep copied in the first place, but maybe that's just evidence that I'm not sure how useful this is in the first place.. or that it's not clear to me what the point was in the first place :-).
agreed.
The question then is: what's a good name for my "mumble_mumble" argument? Does "deep=False" make sense? Or is it "recurse=False"?
When this thread started, I didn't know that it was actually deep copying all members, and had suggested "deep" and "shallow" as analogous to copying. But as it really is doing copying, so i think "deep" and "shallow" are fine. Which leaves us with ``deep=False`` or ``shallow=True``. I usually prefer the default to be True, but in this case, I still like to thing of "deep" as the special case, so I'd go with: ``deep=False``. or ``deep_copy=False``.
well, yes.
The target is json, but I'm not sure how well it fills that need.
If you ask me, not very well, and there would have been no need to copy lists and the like for that purpose anyway. JSON-compatible Python is a specific thing -- you are either targeting that or not :-)
agreed on both points :-) One thing that *might* be useful is a recursing on dataclasses, but not copying anything else :-) -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
participants (3)
-
Anton Abrosimov -
Christopher Barker -
Eric V. Smith