On Fri, Jul 17, 2020 at 7:21 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Fri, Jul 17, 2020 at 11:11:17AM -0400, Ricky Teachey wrote:


For backwards-compatibility, there will only ever be a single positional
argument passed into the method. That's because comma-separated values
in a subscript are already passed as a tuple:

    # this calls __getitem__ with a single tuple argument
    obj[a,b:c,d]  ==> (1, slice(2, 3), 4)

So that's not going to change (at least not without a long and painful
deprecation process). But adding support for keyword arguments requires
no changes to any existing class or a new builtin "key object" type.

This strikes me as problematic for having a consistent mental model of how stuff works in python. I think that for many the difference in the meaning of the syntax between item-getting/setting and function-calling would be... glaring.

The left-hand-side syntax below looks pretty much identical in both cases (except for the sugar of the slice object), but it feels surprisingly different meanings for the positional arguments:

>>> a, b, c, d = (1, 2, 3, 4)
>>> args = a, slice(b, c), d
>>> obj[a, b:c, d, e=5, f=6]  == obj.__getitem__(args, e=5, f=6)
>>> f(a, slice(b,c), d, e=5, f=6)  != f(args, d, e=5, f=6)

On the one hand, a fairly experienced person (who is familiar with the history of the item dunders, and a preexisting mental model that they are always being supplied a single positional argument ) should not have too much of a problem understanding WHY these would behave differently.

But on the other hand, even as an experienced person, this really messes with my brain, looking at it. It's hard for me to believe this isn't going to be a painful distinction for a large number of people to hold in their head-- especially beginners (but not only beginners).

A potentially elegant way around this glaring difference in the meaning of the syntax might be the key-object paradigm Jonathan Fine has suggested. However, that only works if you disallow mixing together positional arguments and kwd args inside the [ ]:

>>> d[a="foo", b="bar"] == d[KeyObject(a="foo", b="bar")]

If you specifically desire to mix together positional and kwd arguments, the key-object paradigm isn't nearly as elegant... there's still magical stuff happening to "split off" the positional arguments from the keyword arguments:

>>> # The positional arguments aren't part of the KeyObject
>>> d[a, b:c, d, e=5, f=6] == d.__getitem__((a, b:c, d), KeyObject(e=5, f=6))

This raises a question that needs to be answered, then: what would be the utility of mixing together positional and kwd arguments in this way?

Even the xarray examples given so far don't seem to make use of this mixture. From my knowledge of pandas I am not sure what the meaning of this would be, either.

> But! On the other hand, the huge difference here is that currently, the
> signature of the get/set item dunders only accepts a single key argument.

To be precise, you can put any signature you like into a `__getitem__`
method, but only the first positional argument will be called from
subscripting syntax.

I appreciate you pointing that out, it's a meaningful distinction because it could mean that rewriting the default __getitem__ and __setitem__ signatures-- as you seem to be arguing for over the key-object paradigm-- won't be as nearly a painful/breaking change as it would be otherwise.
Even if you pass comma-separated values, including slices, they all get
packed into a tuple and passed as the first parameter `item`.

But there is no need to emulate that for keyword args. They can, and I
think should, simply be unpacked into keyword parameters exactly the
same as function call syntax does.

Yup. Personally I really like the idea that unpacking some dict x will work the same as unpacking in a function call:

>>> # Pretty!
>>> d[**x] == d[a=1, b=2]
>>> f(**x] == f(a=1, b=2)

Getting to the end here, I guess I'm really just wondering whether mixing positional and kwd args is worth doing. If it isn't, then the key-object paradigm seems like might be a nicer solution to me for the sole reason that the mental model gets confused otherwise.