Request for comments on PEP 637 - Support for indexing with keyword arguments
Dear all, I would like to know your opinion on how to address a specific need of the new PEP 637: https://github.com/python/peps/blob/master/pep-0637.txt Such PEP would make a syntax like the following valid obj[2, x=23] obj[2, 4, x=23] Which would resolve to a call in the form type(obj).__getitem__(obj, 2, x=23) type(obj).__getitem__(obj, (2, 4), x=23) and similar for set and del. After discussion, we currently have one open point we are unsure how to address, that is what to pass when no positional index is passed, that is: obj[x=23] We are unsure if we should resolve this call with None or the empty tuple in the positional index: type(obj).__getitem__(obj, None, x=23) type(obj).__getitem__(obj, (), x=23) You can see a detailed discussion in the PEP at L913 https://github.com/python/peps/blob/1fb19ac3a57c9793669ea9532fb840861d4d7566... One of the commenters on python-ideas reported that using None might introduce an issue in numpy, as None is used to create new axes, hence the proposal for rejection of None as a solution. However, we are unsure how strongly this would affect numpy and similar packages, as well as what developer will find more natural to receive in that case. We would like to hear your opinion on the topic. Thank you for your help. -- Kind regards, Stefano Borini
On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini
Dear all,
I would like to know your opinion on how to address a specific need of
the new PEP 637:
https://github.com/python/peps/blob/master/pep-0637.txt
Such PEP would make a syntax like the following valid
obj[2, x=23]
obj[2, 4, x=23]
Which would resolve to a call in the form
type(obj).__getitem__(obj, 2, x=23)
type(obj).__getitem__(obj, (2, 4), x=23)
and similar for set and del.
After discussion, we currently have one open point we are unsure how
to address, that is what to pass when no positional index is passed,
that is:
obj[x=23]
We are unsure if we should resolve this call with None or the empty
tuple in the positional index:
type(obj).__getitem__(obj, None, x=23)
type(obj).__getitem__(obj, (), x=23)
Hi Stefano -- thanks for pushing this proposal forward! I am sure that support for keyword indexing will be very welcome in the scientific Python ecosystem. I have not been following the full discussion on PEP 637, but I recall seeing another suggestion earlier for what this could be resolved into: type(obj).__getitem__(obj, x=23) I.e., not passing a positional argument at all. The author of a class that supports keyword indexing could check this sort of case with a positional only argument with a default value, e.g., def __getitem__(self, key=MY_SENTINEL, /, **kwargs): where MY_SENTINEL could be any desired sentinel value, including either None or (). Is there a reason for rejecting this option? It seems like a nice explicit alternative to prespecifing the choice of sentinel value. I guess the concern might be that this would not suffice for __setitem__?
You can see a detailed discussion in the PEP at L913
https://github.com/python/peps/blob/1fb19ac3a57c9793669ea9532fb840861d4d7566...
One of the commenters on python-ideas reported that using None might
introduce an issue in numpy, as None is used to create new axes, hence
the proposal for rejection of None as a solution.
However, we are unsure how strongly this would affect numpy and
similar packages, as well as what developer will find more natural to
receive in that case. We would like to hear your opinion on the topic.
I guess the case this would disallow is distinguishing between obj[None, x=23] and obj[x=23]? Yes, this could be a little awkward potentially. The tuple would definitely be more natural for NumPy users, given the that first step of __getitem__/__setitem__ methods in the broader NumPy ecosystem is typically packing non-tuple keys into a tuple, e.g., def __getitem__(self, key): if not isinstance(key, tuple): key = (key,) ... That said: - NumPy itself is unlikely to support keyword indexing anytime soon. - New packages could encourage using explicit aliases like "np.newaxis" instead of "None", which in general is a best practice already. - The combined use of keyword indexing *and* insertion of new axes at the same time strikes me as something that would be unusual in practice. From what I've seen, it is most useful to either use entirely unnamed or entirely named axes. In the later case, I might write something like obj[x=None] to indicate inserting a new dimension with the name "x". I think we could definitely live with it either way. I would lean towards using an empty tuple, but I agree that it feels a little uglier than using None (though perhaps not surprising, given the already ugly special cases for tuples in the indexing protocol). Best, Stephan
On Wed, 2020-09-23 at 21:41 -0700, Stephan Hoyer wrote:
On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini < stefano.borini@gmail.com> wrote:
<snip>
Hi Stefano -- thanks for pushing this proposal forward! I am sure that support for keyword indexing will be very welcome in the scientific Python ecosystem.
I have not been following the full discussion on PEP 637, but I recall seeing another suggestion earlier for what this could be resolved into:
type(obj).__getitem__(obj, x=23)
I.e., not passing a positional argument at all.
The author of a class that supports keyword indexing could check this sort of case with a positional only argument with a default value, e.g.,
def __getitem__(self, key=MY_SENTINEL, /, **kwargs):
where MY_SENTINEL could be any desired sentinel value, including either None or (). Is there a reason for rejecting this option? It seems like a nice explicit alternative to prespecifing the choice of sentinel value.
I guess the concern might be that this would not suffice for __setitem__?
You can see a detailed discussion in the PEP at L913
<snip>
I guess the case this would disallow is distinguishing between obj[None, x=23] and obj[x=23]?
Yes, this could be a little awkward potentially. The tuple would definitely be more natural for NumPy users, given the that first step of __getitem__/__setitem__ methods in the broader NumPy ecosystem is typically packing non-tuple keys into a tuple, e.g.,
def __getitem__(self, key): if not isinstance(key, tuple): key = (key,) ...
That said: - NumPy itself is unlikely to support keyword indexing anytime soon. - New packages could encourage using explicit aliases like "np.newaxis" instead of "None", which in general is a best practice already. - The combined use of keyword indexing *and* insertion of new axes at the same time strikes me as something that would be unusual in practice. From what I've seen, it is most useful to either use entirely unnamed or entirely named axes. In the later case, I might write something like obj[x=None] to indicate inserting a new dimension with the name "x".
Just to briefly second these points and the general support thanks for the hard work! I do also wonder about the `key=custom_default` solution, or whether there may be other option to address this. For NumPy, I do hope we can play with the idea of adding support if this PEP lands. But agree that labeled axes in NumPy is unlikely to be on the immediate horizon, and I am not sure how feasible it is. My main in the discussion on python-ideas was `arr[]` doing something unexpected, but it was long decided that it will remain a SyntaxError. For the question at hand, it seems to me that mixing labeled and unlabeled indexing would be an error for array-like objects. In that case, the worst we get seems a quirk where `arr[None, x=4]` is not an error, when it should be an error. That does not really strike me as a blocker. I am not a fan of such quirks. But some trade-off seems unavoidable considering the backward compatibility constraints and differences between array-likes and typing use of `__getitem__`. - Sebastian
I think we could definitely live with it either way. I would lean towards using an empty tuple, but I agree that it feels a little uglier than using None (though perhaps not surprising, given the already ugly special cases for tuples in the indexing protocol).
Best, Stephan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
This PEP also opens the possibility of allowing a[] with nothing in the
getitem. Has that been considered?
Aaron Meurer
On Thu, Sep 24, 2020 at 1:48 PM Sebastian Berg
On Wed, 2020-09-23 at 21:41 -0700, Stephan Hoyer wrote:
On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini < stefano.borini@gmail.com> wrote:
<snip>
Hi Stefano -- thanks for pushing this proposal forward! I am sure that support for keyword indexing will be very welcome in the scientific Python ecosystem.
I have not been following the full discussion on PEP 637, but I recall seeing another suggestion earlier for what this could be resolved into:
type(obj).__getitem__(obj, x=23)
I.e., not passing a positional argument at all.
The author of a class that supports keyword indexing could check this sort of case with a positional only argument with a default value, e.g.,
def __getitem__(self, key=MY_SENTINEL, /, **kwargs):
where MY_SENTINEL could be any desired sentinel value, including either None or (). Is there a reason for rejecting this option? It seems like a nice explicit alternative to prespecifing the choice of sentinel value.
I guess the concern might be that this would not suffice for __setitem__?
You can see a detailed discussion in the PEP at L913
<snip>
I guess the case this would disallow is distinguishing between obj[None, x=23] and obj[x=23]?
Yes, this could be a little awkward potentially. The tuple would definitely be more natural for NumPy users, given the that first step of __getitem__/__setitem__ methods in the broader NumPy ecosystem is typically packing non-tuple keys into a tuple, e.g.,
def __getitem__(self, key): if not isinstance(key, tuple): key = (key,) ...
That said: - NumPy itself is unlikely to support keyword indexing anytime soon. - New packages could encourage using explicit aliases like "np.newaxis" instead of "None", which in general is a best practice already. - The combined use of keyword indexing *and* insertion of new axes at the same time strikes me as something that would be unusual in practice. From what I've seen, it is most useful to either use entirely unnamed or entirely named axes. In the later case, I might write something like obj[x=None] to indicate inserting a new dimension with the name "x".
Just to briefly second these points and the general support thanks for the hard work! I do also wonder about the `key=custom_default` solution, or whether there may be other option to address this.
For NumPy, I do hope we can play with the idea of adding support if this PEP lands. But agree that labeled axes in NumPy is unlikely to be on the immediate horizon, and I am not sure how feasible it is.
My main in the discussion on python-ideas was `arr[]` doing something unexpected, but it was long decided that it will remain a SyntaxError.
For the question at hand, it seems to me that mixing labeled and unlabeled indexing would be an error for array-like objects. In that case, the worst we get seems a quirk where `arr[None, x=4]` is not an error, when it should be an error. That does not really strike me as a blocker.
I am not a fan of such quirks. But some trade-off seems unavoidable considering the backward compatibility constraints and differences between array-likes and typing use of `__getitem__`.
- Sebastian
I think we could definitely live with it either way. I would lean towards using an empty tuple, but I agree that it feels a little uglier than using None (though perhaps not surprising, given the already ugly special cases for tuples in the indexing protocol).
Best, Stephan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
I agree with Stephan's suggestion of having no default value for positional
indices, and letting the user supply it.
It seems I replied badly to the mail on the python-ideas list, and my
response ended up as a separate thread, at
https://mail.python.org/archives/list/python-ideas@python.org/thread/ZQ4CHT3...
.
Eric
On Thu, 24 Sep 2020 at 22:22, Aaron Meurer
This PEP also opens the possibility of allowing a[] with nothing in the getitem. Has that been considered?
Aaron Meurer
On Thu, Sep 24, 2020 at 1:48 PM Sebastian Berg
wrote: On Wed, 2020-09-23 at 21:41 -0700, Stephan Hoyer wrote:
On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini < stefano.borini@gmail.com> wrote:
<snip>
Hi Stefano -- thanks for pushing this proposal forward! I am sure that support for keyword indexing will be very welcome in the scientific Python ecosystem.
I have not been following the full discussion on PEP 637, but I recall seeing another suggestion earlier for what this could be resolved into:
type(obj).__getitem__(obj, x=23)
I.e., not passing a positional argument at all.
The author of a class that supports keyword indexing could check this sort of case with a positional only argument with a default value, e.g.,
def __getitem__(self, key=MY_SENTINEL, /, **kwargs):
where MY_SENTINEL could be any desired sentinel value, including either None or (). Is there a reason for rejecting this option? It seems like a nice explicit alternative to prespecifing the choice of sentinel value.
I guess the concern might be that this would not suffice for __setitem__?
You can see a detailed discussion in the PEP at L913
<snip>
I guess the case this would disallow is distinguishing between obj[None, x=23] and obj[x=23]?
Yes, this could be a little awkward potentially. The tuple would definitely be more natural for NumPy users, given the that first step of __getitem__/__setitem__ methods in the broader NumPy ecosystem is typically packing non-tuple keys into a tuple, e.g.,
def __getitem__(self, key): if not isinstance(key, tuple): key = (key,) ...
That said: - NumPy itself is unlikely to support keyword indexing anytime soon. - New packages could encourage using explicit aliases like "np.newaxis" instead of "None", which in general is a best practice already. - The combined use of keyword indexing *and* insertion of new axes at the same time strikes me as something that would be unusual in practice. From what I've seen, it is most useful to either use entirely unnamed or entirely named axes. In the later case, I might write something like obj[x=None] to indicate inserting a new dimension with the name "x".
Just to briefly second these points and the general support thanks for the hard work! I do also wonder about the `key=custom_default` solution, or whether there may be other option to address this.
For NumPy, I do hope we can play with the idea of adding support if this PEP lands. But agree that labeled axes in NumPy is unlikely to be on the immediate horizon, and I am not sure how feasible it is.
My main in the discussion on python-ideas was `arr[]` doing something unexpected, but it was long decided that it will remain a SyntaxError.
For the question at hand, it seems to me that mixing labeled and unlabeled indexing would be an error for array-like objects. In that case, the worst we get seems a quirk where `arr[None, x=4]` is not an error, when it should be an error. That does not really strike me as a blocker.
I am not a fan of such quirks. But some trade-off seems unavoidable considering the backward compatibility constraints and differences between array-likes and typing use of `__getitem__`.
- Sebastian
I think we could definitely live with it either way. I would lean towards using an empty tuple, but I agree that it feels a little uglier than using None (though perhaps not surprising, given the already ugly special cases for tuples in the indexing protocol).
Best, Stephan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Hello, For consistency with the current API, I’d suggest an empty tuple. arr[] # arr.__setitem__(()) arr[1] # arr.__setitem__(1) (could be the only exception) arr[1, 2] # arr.__setitem__((1, 2)) # NOT a.__setitem__(1, 2), note the missing parantheses arr[*a, **kw] # arr.__setitem__(a, **kw), perhaps a future extension If we do a[] # arr.__setitem__() a[1] # arr.__setitem__(1) a[1, 2] # arr.__setitem__((1, 2)) a[1, 2, x=3] # arr.__setitem__((1, 2), x=3) a[*a, **kw] # arr.__setitem__(a, **kw) But honestly, if I’m being candid, the form I like most is: arr[*a, **kw] # arr.__setitem__(a, kw) arr[*a] # arr.__setitem__(a) arr[**kw] # arr.__setitem__((), kw) This is driven by my love for generality and hatred of special cases. So I’m +1 for empty tuple or the third form. Best regards, Hameer Abbasi -- Sent from Canary (https://canarymail.io)
On Friday, Sep 25, 2020 at 4:22 PM, Eric Wieser
wrote: I agree with Stephan's suggestion of having no default value for positional indices, and letting the user supply it. It seems I replied badly to the mail on the python-ideas list, and my response ended up as a separate thread, at https://mail.python.org/archives/list/python-ideas@python.org/thread/ZQ4CHT3....
Eric On Thu, 24 Sep 2020 at 22:22, Aaron Meurer
wrote: This PEP also opens the possibility of allowing a[] with nothing in the getitem. Has that been considered?
Aaron Meurer On Thu, Sep 24, 2020 at 1:48 PM Sebastian Berg
wrote: On Wed, 2020-09-23 at 21:41 -0700, Stephan Hoyer wrote:
On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini < stefano.borini@gmail.com (mailto:stefano.borini@gmail.com)> wrote:
<snip>
Hi Stefano -- thanks for pushing this proposal forward! I am sure that support for keyword indexing will be very welcome in the scientific Python ecosystem.
I have not been following the full discussion on PEP 637, but I recall seeing another suggestion earlier for what this could be resolved into:
type(obj).__getitem__(obj, x=23)
I.e., not passing a positional argument at all.
The author of a class that supports keyword indexing could check this sort of case with a positional only argument with a default value, e.g.,
def __getitem__(self, key=MY_SENTINEL, /, **kwargs):
where MY_SENTINEL could be any desired sentinel value, including either None or (). Is there a reason for rejecting this option? It seems like a nice explicit alternative to prespecifing the choice of sentinel value.
I guess the concern might be that this would not suffice for __setitem__?
You can see a detailed discussion in the PEP at L913
<snip>
I guess the case this would disallow is distinguishing between obj[None, x=23] and obj[x=23]?
Yes, this could be a little awkward potentially. The tuple would definitely be more natural for NumPy users, given the that first step of __getitem__/__setitem__ methods in the broader NumPy ecosystem is typically packing non-tuple keys into a tuple, e.g.,
def __getitem__(self, key): if not isinstance(key, tuple): key = (key,) ...
That said: - NumPy itself is unlikely to support keyword indexing anytime soon. - New packages could encourage using explicit aliases like "np.newaxis" instead of "None", which in general is a best practice already. - The combined use of keyword indexing *and* insertion of new axes at the same time strikes me as something that would be unusual in practice. From what I've seen, it is most useful to either use entirely unnamed or entirely named axes. In the later case, I might write something like obj[x=None] to indicate inserting a new dimension with the name "x".
Just to briefly second these points and the general support thanks for the hard work! I do also wonder about the `key=custom_default` solution, or whether there may be other option to address this.
For NumPy, I do hope we can play with the idea of adding support if this PEP lands. But agree that labeled axes in NumPy is unlikely to be on the immediate horizon, and I am not sure how feasible it is.
My main in the discussion on python-ideas was `arr[]` doing something unexpected, but it was long decided that it will remain a SyntaxError.
For the question at hand, it seems to me that mixing labeled and unlabeled indexing would be an error for array-like objects. In that case, the worst we get seems a quirk where `arr[None, x=4]` is not an error, when it should be an error. That does not really strike me as a blocker.
I am not a fan of such quirks. But some trade-off seems unavoidable considering the backward compatibility constraints and differences between array-likes and typing use of `__getitem__`.
- Sebastian
I think we could definitely live with it either way. I would lean towards using an empty tuple, but I agree that it feels a little uglier than using None (though perhaps not surprising, given the already ugly special cases for tuples in the indexing protocol).
Best, Stephan _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org (mailto:NumPy-Discussion@python.org) https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org (mailto:NumPy-Discussion@python.org) https://mail.python.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@python.org (mailto:NumPy-Discussion@python.org) https://mail.python.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
participants (6)
-
Aaron Meurer
-
Eric Wieser
-
Hameer Abbasi
-
Sebastian Berg
-
Stefano Borini
-
Stephan Hoyer