Mailman 3 Right way to dataclass -> dict conversion. - Python-ideas

newer
Standard tool for iterating over...

Right way to dataclass -> dict conversion.

older
Re: Standard tool for iterating...

Anton Abrosimov

29 Dec 2020 29 Dec '20

7:34 p.m.

Example task: ``` from dataclasses import dataclass, asdict, fields from typing import List @dataclass class Point: x: int y: int @dataclass class Axes: title: str points: List[Point] def plot(title: str, points: List[Point]): print(title) for point in points: print(f'x={point.x}', f'y={point.y}') axes = Axes('example', [Point(i, i) for i in range(3)]) # Now I need to plot my axes. ``` Way 0: `plot(axes.title, axes.points)` Yes, it works great, but the topic about `dataclass2dict` conversion. Way 1: `plot(**asdict(axes))` Using `asdict` is what comes to mind first. But it is recursive, so it won't work. `asdict`, `astuple`... is an "bad naming" example, I think. Unexpected name for a recursive function. People don't read the documentation, and even when they do, they forget. As an example: https://stackoverflow.com/q/52229521/6146442 Viewed 7k times. Way 2: `plot(**vars(axes))` Everything seems to be good. But pseudo-fields which are `ClassVar` or `InitVar` will be returned. `asdict` use: `return tuple(f for f in fields.values() if f._field_type is _FIELD)` Way 3: Using undocumented `__dataclass_fields__` is look like a hack. Way 4: `plot(**{f.name: getattr(axes, f.name) for f in fields(axes)})` This will actually work. But I'm not sure about the elegance. My suggestions: 1. Add default `True` `recursive` flag to `asdict` and implement Way 4 internally. At the same time, IDE will remind people from stackoverflow that `asdict` is recursive. 2. Or add new not-recursive function to `dataclasses` module. Leave `asdict` frozen. 3. Do nothing. Way 4 is handsome enough. Describe the way in the documentation so that people don't get confused and don't use `vars` or `asdict`. I am ready to offer my assistance in implementation. Thank you for your attention and happy holidays!

Show replies by date

Anton Abrosimov

29 Dec 29 Dec

8:04 p.m.

Way 5: Add default `False` `iter` flag to `dataclass` decorator. ``` def __iter__(self) : return ((f.name, getattr(axes, f.name)) for f in fields(axes)) ``` And use: `plot(**dict(axes))`

Christopher Barker

31 Dec 31 Dec

3:33 a.m.

I think you are getting a bit tangled up in the distinction between what I call "data" and "code": dicts are data dataclasses are code. Granted, in Python this line can be blurry, but I think it's helpful to think about it that way. Thinking of it this way, dataclasses.asdict() is converting code to data, and so it makes sense to convert it completely to data. In many contexts where you want a dict, a custom dataclass (or anything other custom class) as values wouldn't really make sense (imagine passing it into the json lib, for instance). All that being said, an option to do a "shallow" or "deep" conversion in asdict() makes sense to me. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

Eric V. Smith

3:56 p.m.

On 12/29/2020 3:04 PM, Anton Abrosimov wrote:

...

Way 5: Add default `False` `iter` flag to `dataclass` decorator.

``` def __iter__(self) : return ((f.name, getattr(axes, f.name)) for f in fields(axes)) ```

And use: `plot(**dict(axes))`

I'd suggest you just write your own function to do this. I consider adding dataclasses.asdict and .astuple to be mistakes. Eric

Christopher Barker

8:18 p.m.

On Thu, Dec 31, 2020 at 11:51 AM Eric V. Smith wrote:

...

I'd suggest you just write your own function to do this. I consider adding dataclasses.asdict and .astuple to be mistakes.

I agree there -- if you want something that can be used like this -- use a dict in the first place :-) Eric: what do you think about adding a "shallow" (or "deep") flag to dataclasses.asdict() that would then upack only the top-level dataclass? -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

Eric V. Smith

8:59 p.m.

On 12/31/2020 3:18 PM, Christopher Barker wrote:

...

On Thu, Dec 31, 2020 at 11:51 AM Eric V. Smith mailto:eric@trueblade.com> wrote:

I'd suggest you just write your own function to do this. I consider adding dataclasses.asdict and .astuple to be mistakes.

I agree there -- if you want something that can be used like this -- use a dict in the first place :-)

Eric: what do you think about adding a "shallow" (or "deep") flag to dataclasses.asdict() that would then upack only the top-level dataclass?

I'm not opposed to it. Since I can't make asdict/astuple go away, I might as well make them more useful. Eric

Eric V. Smith

1 Jan 1 Jan

12:29 a.m.

On 12/31/2020 3:59 PM, Eric V. Smith wrote:

...

On 12/31/2020 3:18 PM, Christopher Barker wrote:

...
On Thu, Dec 31, 2020 at 11:51 AM Eric V. Smith mailto:eric@trueblade.com> wrote:

I'd suggest you just write your own function to do this. I consider adding dataclasses.asdict and .astuple to be mistakes.

I agree there -- if you want something that can be used like this -- use a dict in the first place :-)

Eric: what do you think about adding a "shallow" (or "deep") flag to dataclasses.asdict() that would then upack only the top-level dataclass?

I'm not opposed to it. Since I can't make asdict/astuple go away, I might as well make them more useful.

Currently a deep copy is made: @dataclass class F: l: Optional[list] f: Optional["F"] mylist = [1, 2, 3] f = F(mylist, None) f1 = F(mylist, f)

...

...
...
f = F(mylist, None) f1 = F(mylist, f)

...

...
...
asdict(f) {'l': [1, 2, 3], 'f': None} asdict(f)['l'] is mylist False

...

...
...
asdict(f1) {'l': [1, 2, 3], 'f': {'l': [1, 2, 3], 'f': None}} asdict(f1)['f']['l'] [1, 2, 3] asdict(f1)['f']['l'] is mylist False

I think what we'd want is to just turn the top-level dataclass in to a dict, but then not recurse _and_ only make a shallow copy. Something like:

...

...
...
asdict(f1, mumble_mumble=False) {'l': [1, 2, 3], 'f': F(l=[1, 2, 3], f=None)} asdict(f1)['l'] is mylist True asdict(f1)['f'] is f True

Since the 'f' member wouldn't be a copy (I hope there's agreement here), I'd think the 'l' member should also not be a copy. The question then is: what's a good name for my "mumble_mumble" argument? Does "deep=False" make sense? Or is it "recurse=False"? Or maybe it should just be a separate function. The intent of doing the shallow copy seems different than asdict() currently: asdict() is designed to turn everything to a dict, recursively. The target is json, but I'm not sure how well it fills that need. As I said, I don't think asdict() is a great example of API design, and I wish I'd left it out. Having some convoluted combination of not recursing but making a deep copy seems wrong to me. Eric

Christopher Barker

6:33 p.m.

On Thu, Dec 31, 2020 at 4:29 PM Eric V. Smith wrote:

...

Eric: what do you think about adding a "shallow" (or "deep") flag to dataclasses.asdict() that would then upack only the top-level dataclass?

Currently a deep copy is made:

Honestly, I'm a bit confused as to why arbitrary types were deep copied in the first place, but maybe that's just evidence that I'm not sure how useful this is in the first place.. or that it's not clear to me what the point was in the first place :-).

...

I think what we'd want is to just turn the top-level dataclass in to a dict, but then not recurse _and_ only make a shallow copy.

Something like:

...
...
...
asdict(f1, mumble_mumble=False) {'l': [1, 2, 3], 'f': F(l=[1, 2, 3], f=None)} asdict(f1)['l'] is mylist True asdict(f1)['f'] is f True

Since the 'f' member wouldn't be a copy (I hope there's agreement here), I'd think the 'l' member should also not be a copy.

agreed.

...

The question then is: what's a good name for my "mumble_mumble" argument? Does "deep=False" make sense? Or is it "recurse=False"?

When this thread started, I didn't know that it was actually deep copying all members, and had suggested "deep" and "shallow" as analogous to copying. But as it really is doing copying, so i think "deep" and "shallow" are fine. Which leaves us with ``deep=False`` or ``shallow=True``. I usually prefer the default to be True, but in this case, I still like to thing of "deep" as the special case, so I'd go with: ``deep=False``. or ``deep_copy=False``.

...

Or maybe it should just be a separate function. The intent of doing the shallow copy seems different than asdict() currently: asdict() is designed to turn everything to a dict, recursively.

well, yes.

...

The target is json, but I'm not sure how well it fills that need.

If you ask me, not very well, and there would have been no need to copy lists and the like for that purpose anyway. JSON-compatible Python is a specific thing -- you are either targeting that or not :-)

...

As I said, I don't think asdict() is a great example of API design, and I wish I'd left it out.

Having some convoluted combination of not recursing but making a deep copy seems wrong to me.

agreed on both points :-) One thing that *might* be useful is a recursing on dataclasses, but not copying anything else :-) -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

1205

Age (days ago)

1208

Last active (days ago)

List overview

Download

7 comments

3 participants

participants (3)

Anton Abrosimov
Christopher Barker
Eric V. Smith

Right way to dataclass -> dict conversion.

tags

participants (3)