Package kwkey and PEP 472 -- Support for indexing with keyword arguments
SUMMARY: Some news. I've just published https://pypi.org/project/kwkey/0.0.1/.
This package is about PEP 472 -- Support for indexing with keyword arguments See: https://www.python.org/dev/peps/pep-0472/
As a result, I think we're now in a good position to try this idea out, using present day Python. This includes building some clients that can use the new feature, should it become available. It also includes exploring the design of the API.
The crucial idea is writing >>> from kwkeys import o >>> d[o(1, 2, a=3, b=4)] as a stopgap, until >>> d[1, 2, a=3, b=4] is available.
If you're interested, please do try it out.
IN MORE DETAIL:
On 3 May 2020, Andras Tontas reopened discussion of the PEP. This PEP was created in June 2014, and closed in March 2019, due to lack of interest. See: https://mail.python.org/pipermail/python-dev/2019-March/156693.html
This time round, things went better. On 16 July, Guido wrote (substitute PEP for PRP)
I think it’s a reasonable idea and encourage you to start working on a
design for the API and then a PRP. It would help if someone looked into a prototype implementation as well (once a design has been settled on).
See: https://mail.python.org/archives/list/python-ideas@python.org/message/7MHQYE...
Later on 16 July, I wrote
I'll now state some goals.
- Define 'o' and Protocol so that NOW gives the semantics you wish for.
- Extend Python so that FUTURE give the semantics you wish for.
- And the NOW syntax continues to work as expected (without changing 'o'
and Protocol). 4. And all current use of container[key] continues to work as before.
I believe that it is possible to achieve these goals. My previous posts to this discussion outline some of the key ideas. My next step, when I have time, is to implement and publish general purpose code for the NOW part of this list of goals.
See: https://mail.python.org/archives/list/python-ideas@python.org/message/3IWL42...
On 18 July I wrote that by the end of July I would implement and publish general purpose code for the NOW part of these goals See: https://mail.python.org/archives/list/python-ideas@python.org/message/YW3JXK...
Earlier today, 3 days overdue, I fulfilled my commitment. I hope it helps some of you, and does no harm to others.
On Mon, Aug 3, 2020, 13:11 Jonathan Fine jfine2358@gmail.com wrote:
SUMMARY: Some news. I've just published https://pypi.org/project/kwkey/0.0.1/.
This package is about PEP 472 -- Support for indexing with keyword arguments See: https://www.python.org/dev/peps/pep-0472/
As a result, I think we're now in a good position to try this idea out, using present day Python. This includes building some clients that can use the new feature, should it become available. It also includes exploring the design of the API.
The crucial idea is writing >>> from kwkeys import o >>> d[o(1, 2, a=3, b=4)] as a stopgap, until >>> d[1, 2, a=3, b=4] is available.
If you're interested, please do try it out.
IN MORE DETAIL:
On 3 May 2020, Andras Tontas reopened discussion of the PEP. This PEP was created in June 2014, and closed in March 2019, due to lack of interest. See: https://mail.python.org/pipermail/python-dev/2019-March/156693.html
This time round, things went better. On 16 July, Guido wrote (substitute PEP for PRP)
I think it’s a reasonable idea and encourage you to start working on a
design for the API and then a PRP. It would help if someone looked into a prototype implementation as well (once a design has been settled on).
See: https://mail.python.org/archives/list/python-ideas@python.org/message/7MHQYE...
Later on 16 July, I wrote
I'll now state some goals.
- Define 'o' and Protocol so that NOW gives the semantics you wish for.
- Extend Python so that FUTURE give the semantics you wish for.
- And the NOW syntax continues to work as expected (without changing 'o'
and Protocol). 4. And all current use of container[key] continues to work as before.
I believe that it is possible to achieve these goals. My previous posts to this discussion outline some of the key ideas. My next step, when I have time, is to implement and publish general purpose code for the NOW part of this list of goals.
See: https://mail.python.org/archives/list/python-ideas@python.org/message/3IWL42...
On 18 July I wrote that by the end of July I would implement and publish general purpose code for the NOW part of these goals See: https://mail.python.org/archives/list/python-ideas@python.org/message/YW3JXK...
Earlier today, 3 days overdue, I fulfilled my commitment. I hope it helps some of you, and does no harm to others. -- Jonathan
Do we have an agreement on the API as Guido requested? From my understanding, and please correct me if I am wrong, you are still talking about implementing this using a new class. However, most people who support the use of labelled indexing.and expressed an opinion support a keyword argument-based approach.
I haven't agreed to anything. Though last time I thought about this I was in favor of Steven D'Aprano's idea of translating `x[1, 2, p=3, q=4]` into `x.__getitem__((1, 2), p=3, q=4)`. What the dict class's `__getitem__` would do with that is a different issue -- probably it would be an error.
On Mon, Aug 3, 2020 at 10:32 AM Todd toddrjen@gmail.com wrote:
On Mon, Aug 3, 2020, 13:11 Jonathan Fine jfine2358@gmail.com wrote:
SUMMARY: Some news. I've just published https://pypi.org/project/kwkey/0.0.1/.
This package is about PEP 472 -- Support for indexing with keyword arguments See: https://www.python.org/dev/peps/pep-0472/
As a result, I think we're now in a good position to try this idea out, using present day Python. This includes building some clients that can use the new feature, should it become available. It also includes exploring the design of the API.
The crucial idea is writing >>> from kwkeys import o >>> d[o(1, 2, a=3, b=4)] as a stopgap, until >>> d[1, 2, a=3, b=4] is available.
If you're interested, please do try it out.
IN MORE DETAIL:
On 3 May 2020, Andras Tontas reopened discussion of the PEP. This PEP was created in June 2014, and closed in March 2019, due to lack of interest. See: https://mail.python.org/pipermail/python-dev/2019-March/156693.html
This time round, things went better. On 16 July, Guido wrote (substitute PEP for PRP)
I think it’s a reasonable idea and encourage you to start working on a
design for the API and then a PRP. It would help if someone looked into a prototype implementation as well (once a design has been settled on).
See: https://mail.python.org/archives/list/python-ideas@python.org/message/7MHQYE...
Later on 16 July, I wrote
I'll now state some goals.
- Define 'o' and Protocol so that NOW gives the semantics you wish for.
- Extend Python so that FUTURE give the semantics you wish for.
- And the NOW syntax continues to work as expected (without changing
'o' and Protocol). 4. And all current use of container[key] continues to work as before.
I believe that it is possible to achieve these goals. My previous posts to this discussion outline some of the key ideas. My next step, when I have time, is to implement and publish general purpose code for the NOW part of this list of goals.
See: https://mail.python.org/archives/list/python-ideas@python.org/message/3IWL42...
On 18 July I wrote that by the end of July I would implement and publish general purpose code for the NOW part of these goals See: https://mail.python.org/archives/list/python-ideas@python.org/message/YW3JXK...
Earlier today, 3 days overdue, I fulfilled my commitment. I hope it helps some of you, and does no harm to others. -- Jonathan
Do we have an agreement on the API as Guido requested? From my understanding, and please correct me if I am wrong, you are still talking about implementing this using a new class. However, most people who support the use of labelled indexing.and expressed an opinion support a keyword argument-based approach. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VCFLB4... Code of Conduct: http://python.org/psf/codeofconduct/
I wasn't saying you had agreed to anything, I was saying you requested that others agree on an API before trying to implement a prototype of it ("It would help if someone looked into a prototype implementation as well (once a design has been settled on).").
Jonathan seems to be going ahead with his own API (using a new class to hold all indexes, named or otherwise), which most people who commented didn't seem to like.
On Mon, Aug 3, 2020, 13:43 Guido van Rossum guido@python.org wrote:
I haven't agreed to anything. Though last time I thought about this I was in favor of Steven D'Aprano's idea of translating `x[1, 2, p=3, q=4]` into `x.__getitem__((1, 2), p=3, q=4)`. What the dict class's `__getitem__` would do with that is a different issue -- probably it would be an error.
On Mon, Aug 3, 2020 at 10:32 AM Todd toddrjen@gmail.com wrote:
On Mon, Aug 3, 2020, 13:11 Jonathan Fine jfine2358@gmail.com wrote:
SUMMARY: Some news. I've just published https://pypi.org/project/kwkey/0.0.1/.
This package is about PEP 472 -- Support for indexing with keyword arguments See: https://www.python.org/dev/peps/pep-0472/
As a result, I think we're now in a good position to try this idea out, using present day Python. This includes building some clients that can use the new feature, should it become available. It also includes exploring the design of the API.
The crucial idea is writing >>> from kwkeys import o >>> d[o(1, 2, a=3, b=4)] as a stopgap, until >>> d[1, 2, a=3, b=4] is available.
If you're interested, please do try it out.
IN MORE DETAIL:
On 3 May 2020, Andras Tontas reopened discussion of the PEP. This PEP was created in June 2014, and closed in March 2019, due to lack of interest. See: https://mail.python.org/pipermail/python-dev/2019-March/156693.html
This time round, things went better. On 16 July, Guido wrote (substitute PEP for PRP)
I think it’s a reasonable idea and encourage you to start working on a
design for the API and then a PRP. It would help if someone looked into a prototype implementation as well (once a design has been settled on).
See: https://mail.python.org/archives/list/python-ideas@python.org/message/7MHQYE...
Later on 16 July, I wrote
I'll now state some goals.
- Define 'o' and Protocol so that NOW gives the semantics you wish for.
- Extend Python so that FUTURE give the semantics you wish for.
- And the NOW syntax continues to work as expected (without changing
'o' and Protocol). 4. And all current use of container[key] continues to work as before.
I believe that it is possible to achieve these goals. My previous posts to this discussion outline some of the key ideas. My next step, when I have time, is to implement and publish general purpose code for the NOW part of this list of goals.
See: https://mail.python.org/archives/list/python-ideas@python.org/message/3IWL42...
On 18 July I wrote that by the end of July I would implement and publish general purpose code for the NOW part of these goals See: https://mail.python.org/archives/list/python-ideas@python.org/message/YW3JXK...
Earlier today, 3 days overdue, I fulfilled my commitment. I hope it helps some of you, and does no harm to others. -- Jonathan
Do we have an agreement on the API as Guido requested? From my understanding, and please correct me if I am wrong, you are still talking about implementing this using a new class. However, most people who support the use of labelled indexing.and expressed an opinion support a keyword argument-based approach. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VCFLB4... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/
One potential issue with this approach just occurred to me. I apologise if my thinking on this is wrong.
Say a project implements __getitem__ with the signature
__getitem__(self, index, **kwargs)
Doesn't that mean that a "index" will not be an allowable index label, and that this conflict will depend on knowing the particular implementation details of the dunder methods?
On Mon, Aug 3, 2020, 13:43 Guido van Rossum guido@python.org wrote:
I haven't agreed to anything. Though last time I thought about this I was in favor of Steven D'Aprano's idea of translating `x[1, 2, p=3, q=4]` into `x.__getitem__((1, 2), p=3, q=4)`. What the dict class's `__getitem__` would do with that is a different issue -- probably it would be an error.
On Mon, Aug 3, 2020 at 10:32 AM Todd toddrjen@gmail.com wrote:
On Mon, Aug 3, 2020, 13:11 Jonathan Fine jfine2358@gmail.com wrote:
SUMMARY: Some news. I've just published https://pypi.org/project/kwkey/0.0.1/.
This package is about PEP 472 -- Support for indexing with keyword arguments See: https://www.python.org/dev/peps/pep-0472/
As a result, I think we're now in a good position to try this idea out, using present day Python. This includes building some clients that can use the new feature, should it become available. It also includes exploring the design of the API.
The crucial idea is writing >>> from kwkeys import o >>> d[o(1, 2, a=3, b=4)] as a stopgap, until >>> d[1, 2, a=3, b=4] is available.
If you're interested, please do try it out.
IN MORE DETAIL:
On 3 May 2020, Andras Tontas reopened discussion of the PEP. This PEP was created in June 2014, and closed in March 2019, due to lack of interest. See: https://mail.python.org/pipermail/python-dev/2019-March/156693.html
This time round, things went better. On 16 July, Guido wrote (substitute PEP for PRP)
I think it’s a reasonable idea and encourage you to start working on a
design for the API and then a PRP. It would help if someone looked into a prototype implementation as well (once a design has been settled on).
See: https://mail.python.org/archives/list/python-ideas@python.org/message/7MHQYE...
Later on 16 July, I wrote
I'll now state some goals.
- Define 'o' and Protocol so that NOW gives the semantics you wish for.
- Extend Python so that FUTURE give the semantics you wish for.
- And the NOW syntax continues to work as expected (without changing
'o' and Protocol). 4. And all current use of container[key] continues to work as before.
I believe that it is possible to achieve these goals. My previous posts to this discussion outline some of the key ideas. My next step, when I have time, is to implement and publish general purpose code for the NOW part of this list of goals.
See: https://mail.python.org/archives/list/python-ideas@python.org/message/3IWL42...
On 18 July I wrote that by the end of July I would implement and publish general purpose code for the NOW part of these goals See: https://mail.python.org/archives/list/python-ideas@python.org/message/YW3JXK...
Earlier today, 3 days overdue, I fulfilled my commitment. I hope it helps some of you, and does no harm to others. -- Jonathan
Do we have an agreement on the API as Guido requested? From my understanding, and please correct me if I am wrong, you are still talking about implementing this using a new class. However, most people who support the use of labelled indexing.and expressed an opinion support a keyword argument-based approach. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VCFLB4... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/
On Tue, Aug 4, 2020 at 4:38 AM Todd toddrjen@gmail.com wrote:
One potential issue with this approach just occurred to me. I apologise if my thinking on this is wrong.
Say a project implements __getitem__ with the signature
__getitem__(self, index, **kwargs)
Doesn't that mean that a "index" will not be an allowable index label, and that this conflict will depend on knowing the particular implementation details of the dunder methods?
Yes, that would be correct. However, the function could instead be defined as:
def __getitem__(self, index, /, **kwargs): ...
and then there'd be no conflict (as "self" and "index" must be passed positionally). In effect, the naive spelling (which permits self and index to be passed as keywords) would be a subtle bug that could easily be corrected.
ChrisA
Yes, that would be correct. However, the function could instead be defined as:
def __getitem__(self, index, /, **kwargs): ...
and then there'd be no conflict (as "self" and "index" must be passed positionally). In effect, the naive spelling (which permits self and index to be passed as keywords) would be a subtle bug that could easily be corrected.
sure, but it would be a bug in a LOT of existing code!
I wonder, if this were to be introduced, if the interpreter could have a special case that would call __getitem__ in a special way to avoid this bug in old code.
-CHB
On Mon, Aug 3, 2020 at 1:49 PM Christopher Barker pythonchb@gmail.com wrote:
Yes, that would be correct. However, the function could instead be defined
as:
def __getitem__(self, index, /, **kwargs): ...
and then there'd be no conflict (as "self" and "index" must be passed positionally). In effect, the naive spelling (which permits self and index to be passed as keywords) would be a subtle bug that could easily be corrected.
sure, but it would be a bug in a LOT of existing code!
I wonder, if this were to be introduced, if the interpreter could have a special case that would call __getitem__ in a special way to avoid this bug in old code.
Good edge case to consider. But would it really be such a problem? If you have an existing class like this:
class C: def __getitem__(self, index): ...
c = C()
then presumably calling `c[1, index=2]` would just be an error (since it would be like attempting to call the method with two values for the `index` argument), and ditto for `c[1, 2, 3, index=4]`. The only odd case might be `c[index=1]` -- but presumably that would be equivalent to `c[(), index=1]` so it would still fail.
On Mon, Aug 3, 2020, 17:13 Guido van Rossum guido@python.org wrote:
On Mon, Aug 3, 2020 at 1:49 PM Christopher Barker pythonchb@gmail.com wrote:
Yes, that would be correct. However, the function could instead be
defined as:
def __getitem__(self, index, /, **kwargs): ...
and then there'd be no conflict (as "self" and "index" must be passed positionally). In effect, the naive spelling (which permits self and index to be passed as keywords) would be a subtle bug that could easily be corrected.
sure, but it would be a bug in a LOT of existing code!
I wonder, if this were to be introduced, if the interpreter could have a special case that would call __getitem__ in a special way to avoid this bug in old code.
Good edge case to consider. But would it really be such a problem? If you have an existing class like this:
class C: def __getitem__(self, index): ... c = C()then presumably calling `c[1, index=2]` would just be an error (since it would be like attempting to call the method with two values for the `index` argument), and ditto for `c[1, 2, 3, index=4]`. The only odd case might be `c[index=1]` -- but presumably that would be equivalent to `c[(), index=1]` so it would still fail.
The problem is that knowing what labels are not allowed would require knowing the implementation details of the dunder methods of each individual class, which I don't think is typically considered public API as far as I am aware.
On Mon, Aug 3, 2020 at 2:35 PM Todd toddrjen@gmail.com wrote:
On Mon, Aug 3, 2020, 17:13 Guido van Rossum guido@python.org wrote:
On Mon, Aug 3, 2020 at 1:49 PM Christopher Barker pythonchb@gmail.com wrote:
Yes, that would be correct. However, the function could instead be
defined as:
def __getitem__(self, index, /, **kwargs): ...
and then there'd be no conflict (as "self" and "index" must be passed positionally). In effect, the naive spelling (which permits self and index to be passed as keywords) would be a subtle bug that could easily be corrected.
sure, but it would be a bug in a LOT of existing code!
I wonder, if this were to be introduced, if the interpreter could have a special case that would call __getitem__ in a special way to avoid this bug in old code.
Good edge case to consider. But would it really be such a problem? If you have an existing class like this:
class C: def __getitem__(self, index): ... c = C()then presumably calling `c[1, index=2]` would just be an error (since it would be like attempting to call the method with two values for the `index` argument), and ditto for `c[1, 2, 3, index=4]`. The only odd case might be `c[index=1]` -- but presumably that would be equivalent to `c[(), index=1]` so it would still fail.
The problem is that knowing what labels are not allowed would require knowing the implementation details of the dunder methods of each individual class, which I don't think is typically considered public API as far as I am aware.
I don't understand what you're arguing; I think you didn't follow my argument here completely.
My point is that all existing `__getitem__` implementations will raise errors if any keywords are given, even if the keyword happens to correspond to the name of the argument (say, `index`). This is to counter Chris B's concern that if an existing `__getitem__` implementation didn't use the '/' notation to indicate that `self` and `index` are positional, it would have a bug. I claim that the bug will *only* be present when someone adds keyword support to their `__getitem__` method without using the '/'. Since that's not existing code, this "proves" that adding this feature would not introduce a subtle bug in a lot of existing code -- only in carelessly written new (or updated) code.
On 4/08/20 9:12 am, Guido van Rossum wrote:
then presumably calling `c[1, index=2]` would just be an error (since it would be like attempting to call the method with two values for the `index` argument),
Hmmm, does this mean that classes providing index notation would now need to document the name of the parameter they use for the index?
On Fri, Aug 7, 2020 at 6:02 PM Greg Ewing greg.ewing@canterbury.ac.nz wrote:
On 4/08/20 9:12 am, Guido van Rossum wrote:
then presumably calling `c[1, index=2]` would just be an error (since it would be like attempting to call the method with two values for the `index` argument),
Hmmm, does this mean that classes providing index notation would now need to document the name of the parameter they use for the index?
No, the whole point of my message was they wouldn't have to.
Another approach could be too simply pass the labelled indices in a dict as a third/fourth positional argument.
So for indexing
b = arr[1, 2, a=3, b=4]
Instead of
__getitem__(self, (1, 2), a=3, b=4)
Just do
__getitem__(self, (1, 2), {'a': 3, 'b': 4})
On Mon, Aug 3, 2020, 16:46 Christopher Barker pythonchb@gmail.com wrote:
Yes, that would be correct. However, the function could instead be defined
as:
def __getitem__(self, index, /, **kwargs): ...
and then there'd be no conflict (as "self" and "index" must be passed positionally). In effect, the naive spelling (which permits self and index to be passed as keywords) would be a subtle bug that could easily be corrected.
sure, but it would be a bug in a LOT of existing code!
I wonder, if this were to be introduced, if the interpreter could have a special case that would call __getitem__ in a special way to avoid this bug in old code.
-CHB
-- Christopher Barker, PhD
Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GL3ZVH... Code of Conduct: http://python.org/psf/codeofconduct/
On Mon, Aug 3, 2020 at 5:23 PM Todd toddrjen@gmail.com wrote:
Another approach could be too simply pass the labelled indices in a dict as a third/fourth positional argument.
So for indexing
b = arr[1, 2, a=3, b=4]
Instead of
__getitem__(self, (1, 2), a=3, b=4)
Just do
__getitem__(self, (1, 2), {'a': 3, 'b': 4})
Steven D's reply to me in the previous thread explains why this isn't a great approach; I'll copy and paste the relevant section.
Summary: having to "break up" the positional dictionary is painful, for not much of a benefit.
His comment was actually more specific to Jonathan Fine's key object idea, but it applies just as much to this API suggestion, I think.
Go through the exercise. I have -- I've written Python 2 code that
needed to handle-keyword only arguments, and this was the only way to do
so.
The "only one parameter, which may receive a keyobject" design will have us writing code something like this:
# I want this: def __getitem__(self, item, * a, b, c, d=0) # but have to write this: def def __getitem__(self, item): # Determine whether we got any keyword arguments. if isinstance(item, keyobject): keys = item item = () elif isinstance(item, tuple): # Assuming that all keyword args are at the end; # if there could be more than one keyobject, or if # they could be anywhere in the tuple, this becomes # even more complex. I don't even want to think # about that case. if item and isinstance(item[-1], keyobject): keys = item[-1] item = item[:-1] else: keys = keyobject() # Now extract the parameters from the key object. if 'a' in keys: a = keys.pop('a') else: raise TypeError('missing keyword argument "a"') # same for b and c d = keys.pop('d', 0) # Check for unexpected keywords. if keys: raise TypeError('unexpected keyword')(Any bugs in the above are not intentional.) And now finally we can actually use the keyword parameters and write the method.
However I'll also point out that another idea from Jonathan Fine has the potential to fix both this problem and the key object signature problem, which is what he called a "SIGNATURE CHANGING ADAPTER".
Here's how it goes. First we write class D: @wibble def __setitem__(self, val, u, v, x, y): pass # Or do something.
Next, we define wibble. It will be a SIGNATURE CHANGING ADAPTER. Those who know how to make decorators will, I hope, have little difficulty in defining wibble to do what is required. For this exercise, assume that k.argv = (1, 2), and k.kwargs = dict(x=3, y=4).
--- Ricky.
"I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Mon, Aug 03, 2020 at 05:51:58PM -0400, Ricky Teachey wrote:
However I'll also point out that another idea from Jonathan Fine has the potential to fix both this problem and the key object signature problem, which is what he called a "SIGNATURE CHANGING ADAPTER".
Here's how it goes. First we write class D: @wibble def __setitem__(self, val, u, v, x, y): pass # Or do something.
Next, we define wibble. It will be a SIGNATURE CHANGING ADAPTER. Those who know how to make decorators will, I hope, have little difficulty in defining wibble to do what is required. For this exercise, assume that k.argv = (1, 2), and k.kwargs = dict(x=3, y=4).
Good news! Thanks to Guido's time machine, Python already supports this signature changing adapter for functions and methods. All you have to do is *leave the decorator out*. In fact you don't even have to define it at all. The Python interpreter already knows how to match up keyword arguments like `u=35` and their associated parameters `u`.
So we don't have to duplicate the argument parsing logic of the interpret in a decorator. We just have to let the interpreter do its thing like it already does.
This is (almost) as flexible as you want:
Want your keyword arguments to be optional? Define them with a default. Want them to be mandatory? No default. Want to accept arbitrary keyword arguments? Include a `**kwargs` parameter. Want all of your keywords bundled together? Use only a `**kwargs` parameter.
The only limitation is that subscripting syntax bundles all positional arguments into a single parameter, for reasons lost deep in the mists of time. But that doesn't affect keyword parameters.
On Mon, Aug 03, 2020 at 05:20:25PM -0400, Todd wrote:
Another approach could be too simply pass the labelled indices in a dict as a third/fourth positional argument.
Why would we want to even consider a new approach to handling keyword arguments which applies only to three dunder methods, `__getitem__`, `__setitem__` and `__delitem__`, instead of handling keyword arguments in the same way that every other method handles them?
That's not a rhetorical question. If there is an advantage to making this a special case, instead of just following the same rules as all other methods, I'm open to hearing why it should be treated as a special case.
So for indexing
b = arr[1, 2, a=3, b=4]
Instead of
__getitem__(self, (1, 2), a=3, b=4)
Just do
__getitem__(self, (1, 2), {'a': 3, 'b': 4})
That would be the effect of defining your method with signature:
def __getitem__(self, index, **kwargs)
so if you specifically want all your keyword arguments bundled into a dict, you can easily get it.
On 4/08/20 1:16 pm, Steven D'Aprano wrote:
Why would we want to even consider a new approach to handling keyword arguments which applies only to three dunder methods, `__getitem__`, `__setitem__` and `__delitem__`, instead of handling keyword arguments in the same way that every other method handles them?
These methods are already kind of screwy in that they don't handle *positional* arguments in the usual way -- packing them into a tuple instead of passing them as individual arguments.
I think this is messing up everyone's intuition on how indexing should be extended to incorporate keyword args, or even whether this should be done at all.
The primary issue I was trying to find a way to reliably and clearly avoid conflicts between the index labels and the positional argument names. So if you have:
__getitem__(self, index, **kwargs)
You can't have an index label named "index", because it conflicts with the "index" positional argument.
Apparently that isn't an issue if you structure it like this instead:
__getitem__(self, index, /, **kwargs)
But projects would need to know to do that.
Also, as Greg alluded to, it is more consistent with how the indices are handled currently, which is to use a single variable rather than positional arguments. There may also be performance implications, although I am not certain about that.
On Mon, Aug 3, 2020 at 9:36 PM Steven D'Aprano steve@pearwood.info wrote:
On Mon, Aug 03, 2020 at 05:20:25PM -0400, Todd wrote:
Another approach could be too simply pass the labelled indices in a dict
as
a third/fourth positional argument.
Why would we want to even consider a new approach to handling keyword arguments which applies only to three dunder methods, `__getitem__`, `__setitem__` and `__delitem__`, instead of handling keyword arguments in the same way that every other method handles them?
That's not a rhetorical question. If there is an advantage to making this a special case, instead of just following the same rules as all other methods, I'm open to hearing why it should be treated as a special case.
So for indexing
b = arr[1, 2, a=3, b=4]
Instead of
__getitem__(self, (1, 2), a=3, b=4)
Just do
__getitem__(self, (1, 2), {'a': 3, 'b': 4})
That would be the effect of defining your method with signature:
def __getitem__(self, index, **kwargs)so if you specifically want all your keyword arguments bundled into a dict, you can easily get it.
-- Steven _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RJGNYW... Code of Conduct: http://python.org/psf/codeofconduct/
Thank you all for your posts. I'm busy now and for the next few days, so have little time to respond. Here's some comments and suggestions.
I hope that Andras, Caleb, Stefano, Neil, Joao Bueno, Todd and Stephan will take a special interest in this post. In the previous thread, these people saw that the proposed new syntax d[1, 2, a=3, b=4] would bring benefits to their own particular use of Python. (Apologies for any omitted names or misunderstanding of posts).
I hope the package kwkey shows that it is possible now to write from kwkey import o d[o(1, 2, a=3, b=4)] as a workable present day substitute for the proposed syntax d[1, 2, a=3, b=4]
I think using this can safely go ahead, even though there may be disagreements on the meaning of 'o' and the implementation of classes that take advantage of the new syntax. Indeed, I think going ahead now will contribute to understanding and resolving the disagreements, by creating a shared experience.
I suggest that those who previously suggested uses for the proposed syntax now implement some examples. (I give a list below.) They can do this using my API, Steven's API, or any other API. Or indeed now, using the return value of 'o' directly.
I've started this process with a toy example: https://github.com/jfine2358/python-kwkey/blob/master/kwkey/example_jfine.py
Here are three aspects to the proposed syntax. They are all important, and good design will balance between the various parts and interests.
First, ordinary programmers, who perhaps want d[1, 2] d[x=1, y=2] d[1, y=2] d[y=2, x=1] to all be equivalent, for d a mapping of whose domain is points in the x-y plane. More complicated examples might be found in function annotations (Andras Tantos, Caleb Donovick), quantum chemistry (Stefano Borini), networkx (Neil Girdhar), numpy and pandas (Joao Bueno), xarrary (Todd, Stephan Hoyer).
Second, there are those who implement classes that make use of the proposed syntax.
Third, there are those who implement the extension of Python that allows d[o(1, 2, a=3, b=4)] to be replaced by d[1, 2, 3, 4]
I suggest that those who see benefits in feature produce experimental implementations via kwkey, just as I did in my kwkey.example_jfine. It is possible to do this now, and so have benefits now, in a way that is reasonably future proof regarding implementation of the proposed new syntax.
If you're a user of kwkey, I will have some time available to help you if you want it.
I hope this helps some, and harms none.
I am following, but very swamped with work so I am kind of to the side for a few more days. I am thinking about looking for a sponsor for the PEP, but at this point it's better if I rework the current analysis in light of what you made and the current discussion. Maybe I should open a new PEP?
On Tue, 4 Aug 2020 at 14:26, Jonathan Fine jfine2358@gmail.com wrote:
Thank you all for your posts. I'm busy now and for the next few days, so have little time to respond. Here's some comments and suggestions.
I hope that Andras, Caleb, Stefano, Neil, Joao Bueno, Todd and Stephan will take a special interest in this post. In the previous thread, these people saw that the proposed new syntax d[1, 2, a=3, b=4] would bring benefits to their own particular use of Python. (Apologies for any omitted names or misunderstanding of posts).
I hope the package kwkey shows that it is possible now to write from kwkey import o d[o(1, 2, a=3, b=4)] as a workable present day substitute for the proposed syntax d[1, 2, a=3, b=4]
I think using this can safely go ahead, even though there may be disagreements on the meaning of 'o' and the implementation of classes that take advantage of the new syntax. Indeed, I think going ahead now will contribute to understanding and resolving the disagreements, by creating a shared experience.
I suggest that those who previously suggested uses for the proposed syntax now implement some examples. (I give a list below.) They can do this using my API, Steven's API, or any other API. Or indeed now, using the return value of 'o' directly.
I've started this process with a toy example: https://github.com/jfine2358/python-kwkey/blob/master/kwkey/example_jfine.py
Here are three aspects to the proposed syntax. They are all important, and good design will balance between the various parts and interests.
First, ordinary programmers, who perhaps want d[1, 2] d[x=1, y=2] d[1, y=2] d[y=2, x=1] to all be equivalent, for d a mapping of whose domain is points in the x-y plane. More complicated examples might be found in function annotations (Andras Tantos, Caleb Donovick), quantum chemistry (Stefano Borini), networkx (Neil Girdhar), numpy and pandas (Joao Bueno), xarrary (Todd, Stephan Hoyer).
Second, there are those who implement classes that make use of the proposed syntax.
Third, there are those who implement the extension of Python that allows d[o(1, 2, a=3, b=4)] to be replaced by d[1, 2, 3, 4]
I suggest that those who see benefits in feature produce experimental implementations via kwkey, just as I did in my kwkey.example_jfine. It is possible to do this now, and so have benefits now, in a way that is reasonably future proof regarding implementation of the proposed new syntax.
If you're a user of kwkey, I will have some time available to help you if you want it.
I hope this helps some, and harms none.
Jonathan
On Wed, Aug 5, 2020 at 3:01 PM Stefano Borini stefano.borini@gmail.com wrote:
Maybe I should open a new PEP?
I"ll let teh PEP editors decide, but it look slike it was "rejected"m with this comment:
"The idea never seemed to gain any traction over its near 5 years in existence as a PEP."
So I'd think re-opening it would be fine -- rather than clutter up the PEP namespace...
Maybe we could use a "suspended" status for PEPs?
-CHB
On Tue, 4 Aug 2020 at 14:26, Jonathan Fine jfine2358@gmail.com wrote:
Thank you all for your posts. I'm busy now and for the next few days, so
have little time to respond. Here's some comments and suggestions.
I hope that Andras, Caleb, Stefano, Neil, Joao Bueno, Todd and Stephan
will take a special interest in this post. In the previous thread, these people saw that the proposed new syntax
d[1, 2, a=3, b=4]would bring benefits to their own particular use of Python. (Apologies
for any omitted names or misunderstanding of posts).
I hope the package kwkey shows that it is possible now to write from kwkey import o d[o(1, 2, a=3, b=4)] as a workable present day substitute for the proposed syntax d[1, 2, a=3, b=4]
I think using this can safely go ahead, even though there may be
disagreements on the meaning of 'o' and the implementation of classes that take advantage of the new syntax. Indeed, I think going ahead now will contribute to understanding and resolving the disagreements, by creating a shared experience.
I suggest that those who previously suggested uses for the proposed
syntax now implement some examples. (I give a list below.) They can do this using my API, Steven's API, or any other API. Or indeed now, using the return value of 'o' directly.
I've started this process with a toy example:
https://github.com/jfine2358/python-kwkey/blob/master/kwkey/example_jfine.py
Here are three aspects to the proposed syntax. They are all important,
and good design will balance between the various parts and interests.
First, ordinary programmers, who perhaps want d[1, 2] d[x=1, y=2] d[1, y=2] d[y=2, x=1] to all be equivalent, for d a mapping of whose domain is points in the
x-y plane. More complicated examples might be found in function annotations (Andras Tantos, Caleb Donovick), quantum chemistry (Stefano Borini), networkx (Neil Girdhar), numpy and pandas (Joao Bueno), xarrary (Todd, Stephan Hoyer).
Second, there are those who implement classes that make use of the
proposed syntax.
Third, there are those who implement the extension of Python that allows d[o(1, 2, a=3, b=4)] to be replaced by d[1, 2, 3, 4]
I suggest that those who see benefits in feature produce experimental
implementations via kwkey, just as I did in my kwkey.example_jfine. It is possible to do this now, and so have benefits now, in a way that is reasonably future proof regarding implementation of the proposed new syntax.
If you're a user of kwkey, I will have some time available to help you
if you want it.
I hope this helps some, and harms none.
Jonathan
-- Kind regards,
Stefano Borini _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QK3YV3... Code of Conduct: http://python.org/psf/codeofconduct/
Ok so let's do it like this. I'll open a PR against the PEP and I will aggregate all the feedback from this discussion as additional notes. I'll have to re-read the PEP myself, It's been a while. As I said, I'm swamped so I might start working on it probably on Monday.
On Wed, 5 Aug 2020 at 23:08, Christopher Barker pythonchb@gmail.com wrote:
On Wed, Aug 5, 2020 at 3:01 PM Stefano Borini stefano.borini@gmail.com wrote:
Maybe I should open a new PEP?
I"ll let teh PEP editors decide, but it look slike it was "rejected"m with this comment:
"The idea never seemed to gain any traction over its near 5 years in existence as a PEP."
So I'd think re-opening it would be fine -- rather than clutter up the PEP namespace...
Maybe we could use a "suspended" status for PEPs?
-CHB
On Tue, 4 Aug 2020 at 14:26, Jonathan Fine jfine2358@gmail.com wrote:
Thank you all for your posts. I'm busy now and for the next few days, so have little time to respond. Here's some comments and suggestions.
I hope that Andras, Caleb, Stefano, Neil, Joao Bueno, Todd and Stephan will take a special interest in this post. In the previous thread, these people saw that the proposed new syntax d[1, 2, a=3, b=4] would bring benefits to their own particular use of Python. (Apologies for any omitted names or misunderstanding of posts).
I hope the package kwkey shows that it is possible now to write from kwkey import o d[o(1, 2, a=3, b=4)] as a workable present day substitute for the proposed syntax d[1, 2, a=3, b=4]
I think using this can safely go ahead, even though there may be disagreements on the meaning of 'o' and the implementation of classes that take advantage of the new syntax. Indeed, I think going ahead now will contribute to understanding and resolving the disagreements, by creating a shared experience.
I suggest that those who previously suggested uses for the proposed syntax now implement some examples. (I give a list below.) They can do this using my API, Steven's API, or any other API. Or indeed now, using the return value of 'o' directly.
I've started this process with a toy example: https://github.com/jfine2358/python-kwkey/blob/master/kwkey/example_jfine.py
Here are three aspects to the proposed syntax. They are all important, and good design will balance between the various parts and interests.
First, ordinary programmers, who perhaps want d[1, 2] d[x=1, y=2] d[1, y=2] d[y=2, x=1] to all be equivalent, for d a mapping of whose domain is points in the x-y plane. More complicated examples might be found in function annotations (Andras Tantos, Caleb Donovick), quantum chemistry (Stefano Borini), networkx (Neil Girdhar), numpy and pandas (Joao Bueno), xarrary (Todd, Stephan Hoyer).
Second, there are those who implement classes that make use of the proposed syntax.
Third, there are those who implement the extension of Python that allows d[o(1, 2, a=3, b=4)] to be replaced by d[1, 2, 3, 4]
I suggest that those who see benefits in feature produce experimental implementations via kwkey, just as I did in my kwkey.example_jfine. It is possible to do this now, and so have benefits now, in a way that is reasonably future proof regarding implementation of the proposed new syntax.
If you're a user of kwkey, I will have some time available to help you if you want it.
I hope this helps some, and harms none.
Jonathan
-- Kind regards,
Stefano Borini _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QK3YV3... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD
Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
On 4/08/20 9:20 am, Todd wrote:
Another approach could be too simply pass the labelled indices in a dict as a third/fourth positional argument.
Has anyone suggested attaching the keyword args as attributes on the slice object?
This would avoid changing the signature of __getitem__, and existing __getitem__ methods would just ignore the keyword args.
Hi Todd
You wrote:
Do we have an agreement on the API as Guido requested? From my understanding, and please correct me if I am wrong, you are still talking about implementing this using a new class. However, most people who support the use of labelled indexing.and expressed an opinion support a keyword argument-based approach.
Thank you for your prompt and close interest. For clarity, I don't think we have agreement on the API. (While I was writing this message, Guido posted a similar view.)
I see my kwargs package as a way of exploring the design of the API. To help, let me give you a guided tour of some of the files in the package.
You'll find the following code
from .jfine import key_to_jfine
Point = namedtuple('Point', ('x', 'y')) class PointMap(dict): @key_to_jfine def __setitem__(self, val, x, y): pnt = Point(x, y) dict.__setitem__(self, pnt, val)
in https://github.com/jfine2358/python-kwkey/blob/master/kwkey/example_jfine.py
The decorator key_to_jfine comes from the kwkey.jfine submodule. It implements the API that I favour. This is my first contribution to the exploration of the API.
The kwkey.sdaprano implements, I believe, the API that Steven D'Aprano favours. I believe that from this he (or anyone else such as you) can similarly implement example_sdaprano, example_todd and so forth.
Also implemented are kwkeys.test_jfine and kwkey.test_sdaprano submodules. Although a bit cryptic, they do show what is going on.
For example, jfine gives # Arguments passed to setitem. >>> d[o(1, 2, a=3, b=4)] = 'val' (('val', 1, 2), {'a': 3, 'b': 4}) while sdaprano gives # Arguments passed to setitem. >>> d[o(1, 2, a=3, b=4)] = 'val' ((1, 2), 'val', {'a': 3, 'b': 4})
In case it's needed, the current tree, referred to above, is https://github.com/jfine2358/python-kwkey/tree/81817ec01a3f0dbb46897df3f6d81...
Many thanks to Ricky Teachey, who discovered the resources above, and who kindly wrote
I'm really interested in this and it's very helpful that you included two
versions of the proposed API for people to try out: the jfine version (with the kw object) and the stevedaprano version (with kwd arguments passed directly to the dunder methods).
This is I think nicely explained. (Nitpick. I called it sdaprano.py, to align with jfine.py.)
I think it would be really helpful if Steven could confirm or correct my implementation of his proposed API. I think it would also be really helpful to have an example that uses his API. It needn't be the same example as the one I've written.
I hope this helps (and does no harm).
On Mon, Aug 3, 2020 at 1:58 PM Jonathan Fine jfine2358@gmail.com wrote:
The decorator key_to_jfine comes from the kwkey.jfine submodule. It implements the API that I favour. This is my first contribution to the exploration of the API.
I guess the thing I don't understand is why you favor that API. Could you please explain what you think are the advantages of your approach, ideally with some examples where you think your approach is clearer?
Hi Todd
You wrote:
I guess the thing I don't understand is why you favor that API. Could you
please explain what you think are the advantages of your approach, ideally with some examples where you think your approach is clearer?
I've created a github issue for this, which I'll answer here. https://github.com/jfine2358/python-kwkey/issues/1
In kwkey, I provide 3 interfaces, namely the 'o' and 'K' interface, the jfine interface, and the sdaprano interface. I will explain my reasons, in that order.
Recall that we wish to emulate d[1, 2, a=3, b=4] in current Python. I use d[o(1, 2, a=3, b=4)] as a present day syntax that emulates the future syntax. I think this is sufficiently obvious, as to not require explanation.
When there are keyword arguments, 'o' gives an object of a new type, namely 'K'. The reason is backwards compatibility. Suppose instead >>> key = o(1, 2, a=3, b=4) >>> key == ((1, 2), dict(a=3, b=4)) True
Now suppose that the expression d[(1, 2), dict(a=3, b=4)] appears in legacy code. Because it's legacy code, it can't represent d[1, 2, a=3, b=4] in the new syntax. But how can I distinguish it? That's why I introduce a new class 'K'.
It's worth saying that 'o' used the K class only when it has to. This is to give legacy compatibility. Today, d[1, 2] d[(1, 2)] are equivalent.
To finish on 'o' and 'K', so far as I can see, what I've done 1. Provides all information that would be available from a minimal syntax extension. 2. Doesn't provide any information beyond that (hence K used only when necessary). 3. It continues to work with an ordinary dict. 4. The C-Python implementation is minimal (provided adding K isn't too hard).
Now for the jfine interface, implemented by key_to_jfine signature changing decorator. Here I give what I think is a straightforward interface (for the function wrapped by the decorator) that will provide many programmers with an interface that is straightforward to use. In particular, for setitem the signature is fn(self, val, *key.argv, **dict(key.kwargs))
If you don't like the signatures you provide, choose your own signatures, and then if you wish to write a signature changing decorator.
Now for the sdaprano interface, implemented by the key_to_sdaprano signature changing decorator. Here I provide an interface to what is my understanding of Steven's proposal. This is so he and others can use it if they wish.
By the way, as previously noted d[1, 2] d[(1, 2)] are at present equivalent. However, in the new syntax d[1, 2, a=3] d[(1, 2), a=3] are not equivalent. (The first has three arguments, the second two, the first of which is a tuple.)
Finally, as to which interface is the better, I have as the implementer of kwkey tried to be neutral as to the various interfaces.
I hope this helps. If you wish, please ask for more information or help.
I was following you right up till this bit:
By the way, as previously noted d[1, 2] d[(1, 2)] are at present equivalent. However, in the new syntax d[1, 2, a=3] d[(1, 2), a=3] are not equivalent. (The first has three arguments, the second two, the first of which is a tuple.)
In the "New syntax", wouldn't these examples map to:
d[1, 2, a=3] => d.__getitem__((1, 2), a=3) and d[(1, 2), a=3] => d.__getitem__((1, 2), a=3)
I.e. My understanding was that the existing conversion of all positional parameters in a subscript would be packed into a tuple (strictly for consistency with legacy behaviour) while any keywords would be passed as kwargs?
Steve
On 2020-08-06 14:16, Stestagg wrote:
I was following you right up till this bit:
By the way, as previously noted d[1, 2] d[(1, 2)] are at present equivalent. However, in the new syntax d[1, 2, a=3] d[(1, 2), a=3] are not equivalent. (The first has three arguments, the second two, the first of which is a tuple.)In the "New syntax", wouldn't these examples map to:
d[1, 2, a=3] => d.__getitem__((1, 2), a=3) and d[(1, 2), a=3] => d.__getitem__((1, 2), a=3)
Not quite. The second should be:
d[(1, 2), a=3] => d.__getitem__(((1, 2),), a=3)
I.e. My understanding was that the existing conversion of all positional parameters in a subscript would be packed into a tuple (strictly for consistency with legacy behaviour) while any keywords would be passed as kwargs?
Yes fair point.
That seems more correct
On Thu, Aug 6, 2020 at 2:49 PM MRAB python@mrabarnett.plus.com wrote:
On 2020-08-06 14:16, Stestagg wrote:
I was following you right up till this bit:
By the way, as previously noted d[1, 2] d[(1, 2)] are at present equivalent. However, in the new syntax d[1, 2, a=3] d[(1, 2), a=3] are not equivalent. (The first has three arguments, the second two, the first of which is a tuple.)In the "New syntax", wouldn't these examples map to:
d[1, 2, a=3] => d.__getitem__((1, 2), a=3) and d[(1, 2), a=3] => d.__getitem__((1, 2), a=3)
Not quite. The second should be:
d[(1, 2), a=3] => d.__getitem__(((1, 2),), a=3)
I.e. My understanding was that the existing conversion of all positional parameters in a subscript would be packed into a tuple (strictly for consistency with legacy behaviour) while any keywords would be passed as kwargs?
Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/75SBJA... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Aug 06, 2020 at 02:46:01PM +0100, MRAB wrote:
On 2020-08-06 14:16, Stestagg wrote:
In the "New syntax", wouldn't these examples map to:
d[1, 2, a=3] => d.__getitem__((1, 2), a=3) and d[(1, 2), a=3] => d.__getitem__((1, 2), a=3)
That is certainly what I would expect. All positional arguments are packed into a single positional parameter, and any keyword arguments passed separately.
Any current subscript dunder is likely to look like this:
def __getitem__(self, index): # or key, whatever
i.e. it will only have a single positional argument. (Aside from self of course.)
Not quite. The second should be:
d[(1, 2), a=3] => d.__getitem__(((1, 2),), a=3)
Aside from the keyword argument, which is a syntax error, that's not what happens now.
py> class Demo: ... def __getitem__(self, index): ... print(index, type(index)) ... py> d = Demo() py> d[(1, 2)] # Tuple single arg. (1, 2) <class 'tuple'> py> d[1, 2] # Still a tuple. (1, 2) <class 'tuple'>
Adding a keyword arg should not change this.
[Stestagg]
I.e. My understanding was that the existing conversion of all positional parameters in a subscript would be packed into a tuple (strictly for consistency with legacy behaviour) while any keywords would be passed as kwargs?
To be clear, a single argument is not packed into a tuple.
I think that what is actually going on is a matter of precedence. Commas do double duty, separating arguments in calls and creating tuples. Inside a function call, the argument separate takes priority:
func(1, 2, 3)
is interpreted as three arguments separated by commas. If you want a tuple, you need to change the priority by using round brackets:
func((1, 2, 3))
But inside a subscript, there is currently no concept of multiple arguments. There's always only one item, and commas just have their normal meaning of creating a tuple.
I think that's how it works, but I'm not quite good enough at reading the grammar specs to be sure.
https://docs.python.org/3/reference/grammar.html
We are discussing a proposal to extend Python's syntax to allow d[1, 2, a=3, b=4]
We are also discussing the associated semantics. At present d[1, 2] d[(1, 2)] are semantically equivalent.
There is a proposal, that d[1, 2, a=3, b=4] d[(1, 2), a=3, b=4] be semantically equivalent.
I find this troubling, for example because fn(1, 2, a=3, b=4) fn((1, 2), a=3, b=4) are semantically different.
Here's another example. If we are allowed to write d[*argv, a=3, b=4] then the proposal makes this equivalent to d[argv, a=3, b=4] when type(argv) is tuple.
Consider now >>> def fn(*argv): print(argv) >>> argv = 'key' >>> fn(*argv) ('k', 'e', 'y')
I think it would be a trap for the unwary, that the equivalence of d[argv, a=3, b=4] d[*argv, a=3, b=4] depends on the type of argv.
The root of the proposal that d[1, 2, a=3, b=4] d[(1, 2), a=3, b=4] be semantically equivalent is this: At present d[1, 2] d[(1, 2)] are semantically equivalent.
Why not instead , as part of the proposed semantics, make d[1, 2] d[(1, 2)] semantically different, but only for those classes that ask for it. (This would automatically preserve backwards compatibility.)
I believe this is possible and straightforward in the future, and also in the present via the 'o' and K mechanism. I'm happy to implement this in kwkeys, when I have time.
An aside: I also strongly believe that writing and studying examples that use the new syntax, via the 'o' and K mechanism, is essential to making good choices regarding the semantics, the writing and approval of the PEP, and the success of the extension to Python (should the PEP be accepted).
I hope this helps us come to a shared understanding.
On Fri, 7 Aug 2020 at 14:14, Jonathan Fine jfine2358@gmail.com wrote:
At present d[1, 2] d[(1, 2)] are semantically equivalent.
There is a proposal, that d[1, 2, a=3, b=4] d[(1, 2), a=3, b=4] be semantically equivalent.
I find this troubling, for example because fn(1, 2, a=3, b=4) fn((1, 2), a=3, b=4) are semantically different.
I think I've understood your point. To be sure, what's the Steven proposal?
Jonathan, I took a look at your package, and the code, it's a really nice exploration of the solutions.
I did find having to include the o() call in the sdaprano version quite distracting (I realise it's needed to get it to work in current cpython without silly tricks in the interpreter)...
...But I'm not afraid of doing horribly evil things to python for the purposes of proof-of-concepts.
The code in this gist: https://gist.github.com/stestagg/4962d4e86fb586b14138f19af4ae4a02
Implements a very ugly codec hack to make python 'understand' keywords in index constructs, and I use it to show how the approach proposed by Steven D'Aprano (Please correct me if I'm mis-representing anything!) should work in my mind. If you want to run this code yourselves, you can do by downloading the 2 files into a directory, and running `run.py` (with an appropriate PYTHONPATH set)
The bit of the gist that is most relevant is this:
class ObjWithGetitem(Helper):
def __getitem__(self, key=None, foo=None, bar=None): print(f'{key=} {foo=}')
def main(): obj = ObjWithGetitem()
obj[1] # key=1 foo=None
obj[foo=1] # key=None foo=1
obj[1, foo=2] # key=1 foo=2
obj[1, 2] # key=(1, 2) foo=None
obj[1, 2, foo=3] # key=(1, 2) foo=3
obj[(1, 2), foo=3] # key=(1, 2) foo=3
obj[(1, 2), 3, foo=4] # key=((1, 2), 3) foo=4
obj[**{'foo': 9}] # key=None foo=9
obj[1, 2, foo=3, xxx=5] # TypeError: __getitem__() got an unexpected keyword argument 'xxx'
Which I like for its simplicity
I'm not sure about supporting '*args' syntax in indexes at all, unless doing so could be shown to be trivially implementable without any nasty corner-cases.
Steve
On 2020-08-07 13:13, Jonathan Fine wrote:
We are discussing a proposal to extend Python's syntax to allow d[1, 2, a=3, b=4]
We are also discussing the associated semantics. At present d[1, 2] d[(1, 2)] are semantically equivalent.
Python behaves as though it's wrapping parentheses around the index:
d[1, 2] => d[(1, 2)] d[(1, 2)] => d[((1, 2))] == d[(1, 2)]
There is a proposal, that d[1, 2, a=3, b=4] d[(1, 2), a=3, b=4] be semantically equivalent.
Will adding keyword arguments break existing code? No, because they're currently not allowed.
Python doesn't even allow you to unpack with *, so that won't break existing code either.
[snip]
On Mon, Aug 3, 2020 at 1:10 PM Jonathan Fine jfine2358@gmail.com wrote:
SUMMARY: Some news. I've just published https://pypi.org/project/kwkey/0.0.1/.
This package is about PEP 472 -- Support for indexing with keyword arguments See: https://www.python.org/dev/peps/pep-0472/
As a result, I think we're now in a good position to try this idea out, using present day Python. This includes building some clients that can use the new feature, should it become available. It also includes exploring the design of the API.
The crucial idea is writing >>> from kwkeys import o >>> d[o(1, 2, a=3, b=4)] as a stopgap, until >>> d[1, 2, a=3, b=4] is available.
If you're interested, please do try it out.
...
On 18 July I wrote that by the end of July I would implement and publish general purpose code for the NOW part of these goals See: https://mail.python.org/archives/list/python-ideas@python.org/message/YW3JXK...
Earlier today, 3 days overdue, I fulfilled my commitment. I hope it helps some of you, and does no harm to others. -- Jonathan
Thanks Jonathan Fine, I do not think anyone is going to begrudge your being "3 days overdue".
I'm really interested in this and it's very helpful that you included two versions of the proposed API for people to try out: the jfine version (with the kw object) and the stevedaprano version (with kwd arguments passed directly to the dunder methods).
I hope to make time to fiddle around with both APIs-- if I do I'll be sure to provide feedback.
--- Ricky.
"I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
participants (12)
-
Chris Angelico -
Christopher Barker -
Greg Ewing -
Guido van Rossum -
Jonathan Fine -
Marco Sulla -
MRAB -
Ricky Teachey -
Stefano Borini -
Stestagg -
Steven D'Aprano -
Todd