Jonathan thanks for your desire to make PEP 637, and kwd arguments in subscripting in general, as good as possible. I am bowing out of the conversation from this point on though. I am very happy you found my comments helpful and if there is a second competing PEP, I'll be very interested to read it. Rick. On Fri, Oct 23, 2020, 11:13 AM Jonathan Fine wrote:

SUMMARY: I share with you extracts from a personal email Rick Teachey sent me, on my involvement in PEP 637 (keyword arguments in item access), and my answers to some important questions he asks.

BACKGROUND: Last week, prior to my publicly stating my resignation as a co-author of PEP 637, Ricky Teachey sent me a personal email. I found it so deep and perceptive that I asked him for permission to share extracts with you on this list. I'm pleased that he's allowed this, and I'm most grateful to him for this.

The rest of this post consists of extracts from his email, and some brief comments from me.

RICKY'S EMAIL and MY RESPONSES:

Ricky Teachey wrote:

...

It is clear to all-- not a secret-- you take a different view on the desirability of what is currently the favored proposal. It seems to me what has happened here is, Guido, Steven, and now Stefano believe that your involvement in the PEP (one that is championing that proposal you have significant disagreements with) has been undertaken by you out of a motivation to either get it drastically changed, or (again, in their belief) perhaps even to replace it totally with something else.

...

Now I want you to know I don't get that impression from you. I get the impression you joined the PEP team out of a simple desire to make it better, even though you disagreed with parts of it.

PEP 1 says:

...

The PEP author is responsible for building consensus within the community and documenting dissenting opinions.

[The PEP] should clearly explain why the existing language specification

...

is inadequate to address the problem that the PEP solves.

I want keyword arguments in item access to be added to Python, and I want it to be done well. Something that won't cause difficulty 3 to 5 years from now. PEP 1, as above, has good advice that contributes to this goal. These are areas I particularly wanted to contribute to, as a PEP author. I have no wish to use improper methods to obtain what I think is best.

...

Ricky Teachey wrote:

...

I also understand, though, that at least part of the motivation [for developing kwkey] is something else; you believe that if people experiment a bit ahead of time using actual code, the deficiencies of the current proposal will feel worse than people are imaging in their minds. I wouldn't suggest you back down from saying that outright, too. But if you can say so, I would emphasize that the primary purpose is helping people experiment.

At least when convenient, I'm all for experiments prior to making hard-to-reverses changes to Python.

PEP 1 says:

...

For a PEP that adds new functionality or changes language behavior, it is helpful to include a section on how to teach users, new and experienced, how to apply the PEP to their work.

A basic idea of kwkey is an equivalence between d[1, 2, a=3, b=4] = val X(1, 2, a=3, b=4)[d] = val provided X and item keyword access are given the same semantics. This idea will I think help greatly with the 'teach users' section of the PEP.

Observation, experiment and theory are fundamental to science. Astronomy has the observatory. Biology, chemistry and physics have the laboratory. All research scientists publish. We can use kwkey to do experiments, without first having to create a new version of Python. It makes experiments cheap.

My opinions result in predictions for the result of these experiments. But rather than discuss opinions, I'd like us to do experiments and share our experience. I've been surprised by the results of my own experiments using kwkey! We can all learn from experiments, particularly if we're observant.

Ricky Teachey wrote:

...

However, on the other hand, if the first is not really your purpose-- if what you really want is for people to use the package and realize the proposal needs to be changed, I would [...] suggest asking to write a competing PEP, and promote kwkey a great tool that the steering council can use to make a good choice between competing PEPs.

I applaud your suggestion, that the steering council use kwkey as part of making a choice. And I think we're getting closer to having a second PEP.

Here's an idea. At present both of d[1:2] d[1:2, 3:4, 5, 6] are valid syntax, but neither of d[(1:2)] d[(1:2, 3:4, 5, 6)] are valid syntax. This is, I think, a bit of an anomaly.

I suggest a PEP that extended the syntax to allow (1:2) (1:2, 3:4, 5, 6) to be expressions (to be used as function arguments, with the existing item access semantics). I think this is something which most supporters of keyword arguments in item access could support. And which few opponents of keyword arguments in item access would oppose. It's a small and fairly safe change.

And this change would, via constructs such as X((1:2), a=(3:4))[d] = val allow us to get perhaps 75% of the benefits provided by the larger (and more risky) change that would allow d[1:2, a=3:4] = val

I think we could get (1:2) etc expressions into the next version of Python (in 2021). And then, if X((1:2), a=(3:4))[d] = val is widely used, and the semantics of X agreed upon, perhaps d[1:2, a=(3:4)] = val could be added in 2022. My big concern is making a big change now, whose problems emerge in 2023 to 2025.

I hope this helps. And I thank Ricky both for his thoughtful personal email, and for giving me permission to share it with you.

-- Jonathan

Christopher Barker writes:

thing[i,j,k] is exactly equivalent to thing[(i,j,k)] because the tuple is "created by" the parentheses.

Is that correct? As I understand it, the tuple is created by the commas, and the parentheses are basically thrown away by the parser, as usual.

and thing[1:2, 3:4]

does create a tuple, and pass it on to the __*item__ dunder.

and

thing[slice(1,2), slice(1,2)] is exactly equivalent to thing[(slice(1,2), slice(1,2))]

But is it exactly equivalent to "thing[1:2, 1:2]"? I believe it is, and if so

So the only missing piece here is that that slice syntax can only be used directly inside the square brackets.

is exactly correct, and somewhat anomolous, since slices are perfectly good objects.

As the colon is used elsewhere in Python,

As a statement clause separator, which becomes ambiguous: if thing: x However, I would guess this you can resolve this with a grammar that says "if thing:" is unconditionally a conditional clause, and that in "if (thing : x)" has to parse "thing : x" to slice syntax first, then look for the terminating colon for the if clause. Similarly for all other compound statement clauses: an unparenthesized colon terminates the clause immediately. And in dict displays, where generalized slices could be used as components, rather than treating ':' merely as a separator. (I write "generalized" because I presume slices are currently specialized to ints). Is there anything else? I think one big bugaboo here is in the word "generalized", because you'd enable errors like myslice = 'key' : 'value' for item in alist[myslice]: foo(item) and I just don't know if that's plausible enough to worry about. Also, slice syntax admits partial slices, so you would probably need to parse "naked" slices like "'key':", and what in the world would "::'some arbitrary string'" mean, and how could you justify ruling it out arbitrarily? (Wouldn't consenting adults apply, so for example def myzip(slicearg, iterables): myiterables = [itertools.islice(it, slicearg.start, slicearg.stop) for it in iterables] return zip(myiterables, strict=slicearg.step) "should" be allowed?)

I do think it should be completely separate from PEP 637

+1

I suggest a PEP that extended the syntax to allow

...

...

(1:2) (1:2, 3:4, 5, 6) to be expressions

I don't like the requirement for parentheses of a single slice expression, and the second example is presumably not slice syntax, but tuple syntax. I.e., given the requirement exemplified by (1:2), the second example should be written ((1:2), (3:4), 5, 6) which is not as attractive. Except in function syntax, where they are a required operator, elsewhere in Python parentheses are used only for controlling order of evaluation as far as I know, with the partial exception of genexps. I write "partial" because in most places a genexp must be parenthesized, but it's acceptable as the only actual argument to a function without parentheses. And of course a single genexp may contain an arbitrarily large number of clauses, so it's quite different from an n-ary operator with very small n.

A quick googling reveals the rejected PEP 204 https://www.python.org/dev/peps/pep-0204/

which proposed a "range literal", which used slice syntax to create a range -- but that's something different (and, indeed, range objects are different than they were back then as well)

I don't think it's that different, both ranges and slices (as they exist now) are mostly used to control iteration. It's just that the iteration controlled by slices is implicit in the creation of a new sequence. I would guess any arguments purely about the syntax would apply here as well. My current opinion is that (1) generalized slices are just too messy, the constructor notation "slice(start, stop, step)" is not at all obnoxious, and (3) the colons in compound statements, dict displays, and sequence slices are not operators in any of the three cases, and can't be treated that way. Specifically, in all three cases they're syntactically separators, but in compound statements they don't separate sequences of objects at all, and in slices and dict display items the objects they separate are of quite different types and the implied sequences are even of different lengths (3 vs. 2). It's messy enough that I'm not going to spend any more skull sweat on this, we're fine as is for slices and slice syntax (IMO).

On 2020-10-27 23:10, Stephen J. Turnbull wrote:

And in dict displays, where generalized slices could be used as components, rather than treating ':' merely as a separator. (I write "generalized" because I presume slices are currently specialized to ints).

Non-int slices are valid and are used for instance by pandas, where you can do things like `mydataframe.loc['some_row':'other_row', 'some_column':'other_column']` . -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

On Tue, Oct 27, 2020 at 11:24 PM Chris Angelico wrote:

On Wed, Oct 28, 2020 at 5:12 PM Stephen J. Turnbull wrote:

...

...

thing[i,j,k] is exactly equivalent to thing[(i,j,k)] because the tuple is "created by" the parentheses.

Is that correct? As I understand it, the tuple is created by the commas, and the parentheses are basically thrown away by the parser, as usual.

I think that sentence was perhaps meant to read "because the tuple is NOT created by the parentheses" - the rest of the post makes full sense that way.

yes, thanks -- that was a typo, and an unfortunate one, as it's kinda the key point :-( -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython