Add a __valid_getitem_requests__ protocol
It'd be nice to have a __valid_getitem_requests__ protocol that, if present, yields pairs such that: for key, value in items(obj): assert obj[key] == value for any obj. Note that this definition explicitly allows subsets, which makes it useful for sequences and defaultdicts, as they can have all sorts of keys (or even infinitely many in the case of defaultdict):
and as such we should consider limiting the results for at least these 2 cases. I currently use a similar idea in my code but I feel like having this would make my code more extensible without ppl having to monkeypatch it.
On 2020-02-18 5:08 p.m., Rhodri James wrote:
Similar to len(). Just a shitty wrapper for __valid_getitem_requests__. We kinda have a requirement that __iter__ and __contains__ have to be related, see e.g. dict.__iter__ and dict.__contains__. In an ideal world dict would yield pairs by default. and dict() would just iterate whatever. But anyway I digress. As far as my proposal goes, __valid_getitem_requests__ would provide slightly different semantics for converting-to-dict: dict(items(['a', 'b', 'c'])) == {0: 'a', 1: 'b', 2: 'c'} dict(items({1: 'a', 2: 'b'})) == {1: 'a', 2: 'b'} # TODO find other things with __getitem__ and put them here In other words: it'd allow creating a dict whose __getitem__ behaviour is similar to that of the original collection, in the most straightforward case. (note how the first dict doesn't contain negative indices or the infinitely many slice objects that can be used to index a three-element list.) (perhaps a better name for this property is needed? how about "canonical keys"? as in, __valid_getitem_requests__ should contain only the canonical "keys" (the python datamodel specifically says __getitem__ takes a self and a key[1]) of a collection, and their respective values.) [1] https://docs.python.org/3/reference/datamodel.html#object.__getitem__
On 2020-02-18 5:43 p.m., Chris Angelico wrote:
Oh yes absolutely. I have a whole library focused on this sort of "canonical keys"-based iteration: https://ganarchy.autistic.space/project/0f74bd87a23b515b45da7e6f5d9cc8238044... (Friendly warning, it does handle sets in a rather unusual way. I've never used that feature outside unit tests, either. Pay no attention to that however, as it's not relevant to this proposal.)
On Wed, Feb 19, 2020 at 7:58 AM Soni L. <fakedme+py@gmail.com> wrote:
Too short, didn't read. Specifically, I'm not going to go digging through your links to find code to justify why you want this feature. Post some code here, or at very least, link directly to the code that would be improved by this feature. Soni, you're very close to my mental blocklist. Please put a bit more effort into your posts to make it clear what you're actually proposing, rather than making each of us do the work. ChrisA
On Tue, Feb 18, 2020 at 1:06 PM Chris Angelico <rosuav@gmail.com> wrote:
That link leads to a page where there's a link to https://soniex2.autistic.space/git-repos/abdl.git/ which is not accessible without further authentication.
I have to agree with Chris here -- Soni, please put a bit more effort in your posts, if you want to remain welcome here. From your URL I gather you may not have the best social skills. That's fine. We all try to accommodate a diverse audience here. But at some point, if you don't at least show that you're trying your best, you will wear out your welcome here. Maybe that has happened elsewhere. Don't blame it entirely on others or whatever handicap you may have (we have no idea here, except that we observe your posts to be somewhat cryptic). You have started a large number of threads in a short amount of time, and most of them have generated little of use to the Python community. Please think about how you can contribute something more constructive. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On 2020-02-18 6:04 p.m., Chris Angelico wrote:
The documentation states: "ABDL expressions are regex-like constructs for matching and validating object structures. They can be used with JSON and similar formats, and even self-referential data structures." "ABDL expressions have the ability to iterate, index, validate and filter data structures, through the use of the syntax elements below." "An arrow is ``->`` and indicates indexing/iteration (Mappings, Sequences, Sets). Whether indexing or iteration is used is defined by the elements that follow, with iteration being used by default." "A variable is a sequence of alphanumeric characters, not starting with a digit. A ``(key, value)`` tuple containing the respective matched element will be identified by this name in the results dict." And this sums up the core aspect of the library: the library is built on the idea of iterating "canonical keys" and their corresponding values, and validating and filtering them. ("[...] iteration being used by default", meaning indexing takes additional syntax - this is unlike some JSON and XML query languages where indexing takes /less/ syntax and filtering is generally highly limited and verbose.) Iteration is done thusly: def _pairs(obj): if isinstance(obj, collections.abc.Mapping): return iter(obj.items()) elif isinstance(obj, collections.abc.Sequence): return iter(enumerate(obj, 0)) elif isinstance(obj, collections.abc.Set): return iter(((e, e) for e in obj)) else: # maybe there's more stuff I can implement later raise TypeError I think having a proper protocol for this behaviour would be beneficial, as no (monkey)patching would be needed to add support for additional collections and data types. The comment saying there may be additional data types I've missed would then become irrelevant, as they'd be covered by __valid_getitem_requests__. Sometimes you just want to treat everything as a (read-only) dict. Mostly when dealing with user input (e.g. configs), but not only.
On Tue, Feb 18, 2020 at 1:45 PM Soni L. <fakedme+py@gmail.com> wrote:
But what will you do with the pairs? For mappings and sequences, the pairs represent (key, value) or (index, value) pairs, which makes sense, since (as you write in your first post) one can write obj[key] to obtain values from a mapping or sequence (key ~~ index). But then you go on and propose returning pairs (value, value) for sets, and there's nothing corresponding to obj[key]. So I'm not sure what your intent for the proposed is. Why do you want pairs for everything? Suppose obj is a number x, perhaps you would want to get a single pair (x, x) from the iteration? (I know that's probably bending the metaphor too far. :-) Even if you were to answer this question, consider that perhaps your application is rather unique in this insistence on pairs. Or perhaps you should've stopped at proposing an API that unifies mappings and sequences (which more sensible since both support __getitem__) -- isn't that what your original post was about? However even for mappings and sequences the analogy only goes so far, since *iterating* over them behaves very differently: for a mapping, iteration yields keys, while for a sequence, it yields values. A legitimate question that comes up from time to time is actually how to *distinguish* between mappings and sequences -- se the other python-ideas thread that I mentioned previously. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On 2020-02-18 7:03 p.m., Guido van Rossum wrote:
Yep, I'll bring you back to the previous post: "(Friendly warning, it does handle sets in a rather unusual way. I've never used that feature outside unit tests, either. Pay no attention to that however, as it's not relevant to this proposal.)" I kinda just really wish sets did easy interning with a very low chance of accidentally mapping two different things to the same (or otherwise wrong) objects (e.g. intern_cache["a"]=None), and so I decided to encode that into my _pairs. But that is unrelated to this proposal.
I should've stopped myself from adding this weird behaviour with sets. I've never even used it yet. Sequences and dicts, I do use, but not sets.
Yeah I can't say I don't have some "I like the way Lua does this" feelings. I'll be honest tho, if I were to implement fully separate behaviour for lists and dicts, the fact that dicts iterate by key and lists iterate by value means I would have to go with either a slower implementation (such as an operator that basically just converts an iterator of keys into one of values - or at least I'm pretty sure python doesn't optimize "indexing a dict with the key you're iterating", even if that may or may not be a cpython implementation detail) or one with loads of duplicated code (different code for lists and for dicts, and different arrow operators to select between them). I opted for this instead, it seemed like an acceptable tradeoff altho I'm not sure the python community likes it.
On 18/02/2020 20:33, Soni L. wrote:
Language, sunshine. Do you have a use case for this, or is it just theoretically nice to have? I have to say it isn't nice enough for me to actually want it, and I say that as someone who regularly forgets that iterating over a dict gets you its keys. -- Rhodri James *-* Kynesim Ltd
Presumably this was inspired by the other thread, "Add __keys__ or __items__ protocol", where an __items__() method was proposed instead of recognizing mappings by the presence of a keys() method. On Tue, Feb 18, 2020 at 12:46 PM Rhodri James <rhodri@kynesim.co.uk> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Tue, 18 Feb 2020 at 20:47, Rhodri James <rhodri@kynesim.co.uk> wrote:
This looks to me like Lua's `items()` function. It's useful in Lua, but I'm not sure there's any obvious reason to assume it'll be a natural fit for idiomatic Python code. As people have said, do you have a good Python example of real code where this would be useful? (And where dct.items() or enumerate(lst) wouldn't be sufficient - code that expects to work equally with lists or dictionaries seems more Lua-like than Pythonic, IMO). Paul
On Tue, Feb 18, 2020 at 09:04:21PM +0000, Paul Moore wrote:
This looks to me like Lua's `items()` function.
I can't find that function listed anywhere. https://www.lua.org/manual/5.3/ Are you maybe thinking of `pairs()`? https://www.lua.org/manual/5.3/manual.html#pdf-pairs
In Lua, lists (arrays) *are* dictionaries, they keys are merely implicitly set to consecutive integers. My personal experience is that the analogy between (index, item) and (key, item) is more useful in theory and practice, but if I ever needed to do such a thing, a thin helper function: def items(dict_or_seq): try: return obj.items() except AttributeError: return enumerate(obj) would do the job. I don't think this is useful or important enough to turn it into a language-wide protocol. It's not as if the concept has much, if any, use outside of mappings and sequences. -- Steven
On Tue, Feb 18, 2020 at 08:44:07PM +0000, Rhodri James wrote:
Language, sunshine.
What about it? Just for the record, I had no issues with Soni's language, but I do object to attempts to shame him for it. This isn't the Disney Chanel, it's been over 100 years since Victoria was Queen of England, and we're all adults here. -- Steven
On 02/18/2020 04:26 PM, Steven D'Aprano wrote:
On Tue, Feb 18, 2020 at 08:44:07PM +0000, Rhodri James wrote:
Since when is simply drawing one's attention to something shaming?
Really? I didn't realize those under the legal age of majority were not allowed on this list. At any rate, we should be able to control what we say and how we say it, adult or not. -- ~Ethan~
On Feb 18, 2020, at 17:11, Ethan Furman <ethan@stoneleaf.us> wrote:
Isn’t the point of having a Code of Conduct that we shouldn’t have to have these arguments? Or, if there is an argument about whether a single figurative use as a word qualifies as “excessive swearing” per the CoC, surely an -ideas thread isn’t the place to have it?
Yeah, I very much doubt that "shitty wrapper" constitutes as excessive swearing. It'd be one thing if the post contained many or was directed at someone else, but the OP was using it to refer to something they personally wrote. Personally, I think it's a non-issue in that sort of context. On Tue, Feb 18, 2020 at 10:40 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:
I've been sitting on this email for the best part of a week. It hasn't been an easy decision to decide to send it, because I expect it will ruffle some feathers, including someone who, last time I ruffled his feathers, banned me from one of the other mailing lists. But I am part of this community, and if I'm too scared to state my opinions for fear of retaliation, then I'm not really part of the community, the supposed community values of being open and welcoming is a sham, and it's best if I find out now. When I posted my reply to Rhodi, I wasn't intending to shine the spotlight on his comment to Soni more than the simple statement I made. But since Ethan disagreed with me, I should elaborate. On Tue, Feb 18, 2020 at 05:09:39PM -0800, Ethan Furman wrote:
Of course drawing attention to something can often be shaming and a disguised attack: "You're fat. He's filthy. She's promiscuous. Your language is unacceptable and bad (and by implication, so are you)." Having attention drawn to your social transgression is shaming. That's why we do it: to stop people from transgressing again. "Language" is short for "Watch your language", a command, not an observation or even a suggestion. But I'm pretty sure that you, Ethan, already knew that. Defending Rhodi on the basis that he's merely stating a fact is disingenious. In Commonwealth English, calling someone you don't know "sunshine" is a mock-friendly but actually hostile act, rather like the way "pal" or "buddy" can be used in American English. Even if you did not know this, you surely knew that in the context of this email thread, calling someone "sunshine" is condescending, like calling them "pumpkin" or "sweety-pie". What Rhodi did was not "simply drawing one's attention", it was a rebuke. Ethan, I find it difficult to believe that you did not know that, and that your question was made in good faith.
Ethan, you've been involved with the Python community long enough to know the meaning of "we're all adults here", and it doesn't mean anything about legal age of majority. The point is, this is not a child-friendly space, this is an space for adults. We shouldn't be expected to dumb-down our conversation to that suitable for six year olds. If there are any underage people on this list, I expect them to (1) behave like adults, and (2) not freak out when the rest of us behave like adults. And that includes using "grown up words" (dear gods, why are we even having this conversation?) that I am pretty sure most of us, if not all of us, use off-list, hear our peers use, hear at the movies, on television, and on radio. Besides, I know what kids are like. I was one once, as we all were. The idea that an unsupervised child on the Internet doesn't already know this "bad word" by the time they're eight is at best laughably naive, if not patronising and demeaning. When the CoC was first debated, the PSF promised that the CoC would not be used to ban "rude words". Have they reneged on that promise? If not, Soni's language shouldn't even be an issue and Rhodi's rebuke was unnecessary. If they have, then we have a much more serious problem with this community.
At any rate, we should be able to control what we say and how we say it, adult or not.
Indeed we should. And if that means we want to call a shitty piece of software a shitty piece of software, especially when it's our own software, we shouldn't need fear that somebody will tell us off for it. -- Steven
On Mon, Feb 24, 2020 at 10:04 AM Steven D'Aprano <steve@pearwood.info> wrote:
For the record, I think Soni's original description (implying that the thin wrappers like iter(x) calling x.__iter__() etc are in some way bad) was rude, but not a CoC violation or anything. It's the sort of rude that colours someone's posts and thus other people's views of their arguments, but shouldn't get the person banned. But it WAS rude, and I don't think Rhodri was wrong to call that out. If you're going to rail on the code patterns that currently exist (profane language is mostly irrelevant here), that really doesn't do your arguments any favours, and it's fine IMO to call someone out for it just as much as you'd call someone out for posting their entire feature request in all-caps with no punctuation or line breaks. ChrisA
WHATS WRONG WITH TOP POSTING IN ALL CAPS AND NO PUNCTUATION!?!?!?! :) No one has complained to the moderators that CoC is being violated. and if they did, there’s a whole bunch of safeguards to ensure that it’s not being misused. I do occasionally (once or twice a year?) place specific members back on moderate temporarily so that I can intercept heated discussions before they spiral out of control, but I’ve tried to keep moderation rapid enough that I don’t think it’s intrusive. Nothing on this list is time critical, IMO, and I also note that new members are moderated _by default_. Also, if someone thinks that the CoC is being over applied (!?) or the list moderation is too heavy handed (!?) on this list, this list is also not the place to discuss it. That place would be conduct-wg@python.org, and I’m sure they would appreciate any concerns anyone has to convey. best, —titus
This is an honest question: Is it helpful to anyone taking part in this conversation about the conversation to have it? I don’t think it’s helpful to the list as a whole, honestly. I suggest next time the conversation gets a bit heated or offensive, and someone points it out, that everyone involved says “I’m sorry”, then takes a little break, and then the technical discussion can resume. One other suggestion is that if you are finding someone’s posts annoying or offensive, then either block them quietly or send them a direct note explaining why. Having that discussion in public just stirs things up for the rest of us. On the other hand, if folks really do find these conversations about the conversation helpful, maybe start a new thread? -CHB
-- Christopher Barker, PhD
Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On 02/23/2020 03:01 PM, Steven D'Aprano wrote:
The relevant posts from the thread in question: https://mail.python.org/pipermail/python-list/2018-September/887394.html https://mail.python.org/pipermail/python-list/2018-September/887398.html https://mail.python.org/pipermail/python-list/2018-September/887401.html https://mail.python.org/pipermail/python-list/2018-September/887445.html https://mail.python.org/pipermail/python-list/2018-September/887493.html https://mail.python.org/pipermail/python-list/2018-September/887418.html https://mail.python.org/pipermail/python-list/2018-September/887464.html
That's pretty much what you said then. Have you not returned to that mailing list because you no longer feel like part of the community?
When I posted my reply to Rhodri, I wasn't intending to shine the spotlight on his comment to Soni more than the simple statement I made:
(Rhodri has two 'r's in his name.)
But since Ethan disagreed with me, I should elaborate.
Rhodri James wrote: ------------------ Language, sunshine. Steven D'Aprano wrote: --------------------- What about it? Just for the record, I had no issues with Soni's language, but I do object to attempts to shame him for it. Ethan Furman wrote: ------------------ Since when is simply drawing one's attention to something shaming? Steven explains: ---------------
To me, "shaming" also requires a certain amount of meanness or cruelty (the "attack" part), often suggested by the tone or body language of the person commenting. Hopefully we are all aware that tone is hard to achieve in the written word, especially across continents and language barriers; body language is, of course, completely absent.
Had I thought about it that way I would agree with you. I did not, however, see his comment as a command, nor do I agree that "Language" is always short for "Watch your language", especially in the often terse environment of mailing lists.
No, I did not know that. And "pal" and "buddy" are no more hostile than any other American English word unless used in a certain way -- a way made explicit by tone and body language. In fact, when I first read Rhodri's post I thought it meant something along the lines of: programming language -> pie in the sky perfection. Hence, it is possible to be too terse.
I'm sorry you think so poorly of me. Steven previously said: ---------------------- This isn't the Disney Chanel, it's been over 100 years since Victoria was Queen of England, and we're all adults here. Ethan had replied: ----------------- Really? I didn't realize those under the legal age of majority were not allowed on this list.
Yes, it means we are responsible for our own code.
Using words that accurately describe a situation without using profanity is not "dumbing down".
If "behaving like adults" was the accepted norm, we wouldn't have a CoC (and No, I am NOT suggesting that the OP, or this post, are even remotely close to a CoC violation).
And that includes using "grown up words" (dear gods, why are we even having this conversation?)
Because you don't like people disagreeing with you about what is acceptable list behavior?
that I am pretty sure most of us, if not all of us, use off-list, hear our peers use, hear at the movies,on television, and on radio.
Which is irrelevant to what is expected on the Python lists.
It's a matter of courtesy. I, for one, greatly appreciate posters who take the time to formulate their arguments and/or problems without using emotionally charged words and/or expletives which usually add little and can easily take away from the intended conversation (such as your querying of the gods a few moments ago).
When the CoC was first debated, the PSF promised that the CoC would not be used to ban "rude words". Have they reneged on that promise?
I don't know -- maybe you should ask them? Or are you conflating Rhodri with the PSF?
If not, Soni's language shouldn't even be an issue and Rhodri's rebuke was unnecessary.
We'll have to agree to disagree on whether Rhodri's post was a rebuke or a reminder.
If they have, then we have a much more serious problem with this community.
Indeed. I would hate to think that every little disagreement needs to be moderated and judged by the PSF. Ethan wrote: ----------- At any rate, we should be able to control what we say and how we say it, adult or not.
Is this one of those language barrier thingies? Because to me, "telling us off for it" would be quite a bit harsher than the two words Rhodri used. -- ~Ethan~
On Mon, 24 Feb 2020 at 08:52, Ethan Furman <ethan@stoneleaf.us> wrote:
I feel that this discussion has got out of hand, and it would be good for everyone to take a breath and move on. However, *as a data point* I will say that I took Soni's original statement as being a mildly self-deprecating comment about a bit of code they had written (but hadn't, IIRC, actually quoted in the posting, just referred to), which in their view was not particularly clever but got the job done. I personally wouldn't have used the word "shitty" in an email, but I quite likely would have in spoken conversation, and I'm personally much more formal in email than when speaking, and I know many other people aren't. So I have no problem with Soni's use of the word. I took Rhodri's comment as a mild rebuke - worded as it was to be light-hearted and therefore not to be taken *too* seriously. Maybe he doesn't like the language himself, maybe he was enforcing a standard he felt was appropriate for the list, I don't know, and it doesn't really matter. His comment was brief, and full of cultural implications which (as an English person) I suspect I caught but which may have been missed by others who are from different backgrounds. Maybe it was a mistake being quite so colloquial, but it's an easy one to make if you're not spending inordinate amounts of time fine-tuning your emails. Steven's objection to "shaming", while maybe technically accurate, seemed to ignore a lot of the nuance. Maybe that's down to personality, or cultural differences. But I do think it failed to assume (or maybe "allow for the possibility of" would be more accurate) good intentions. But I'm fine with assuming that it was itself a well-intentioned attempt to caution people against giving offense, which didn't have the intended effect. Hopefully, if I've misinterpreted anyone's actual meaning in the above, I've erred on the side of assuming good intention. If anyone feels otherwise, I apologise. I suggest that any harm that might have been done by the original post is getting lost now in the increasing heat of the argument, and we'd be better if we simply let the debate drop at this point. Paul
As the instigator here, I think it's my duty to clarify my intent on a few things. On 24/02/2020 08:49, Ethan Furman wrote:
I have no problems at all with you stating your mind about this, Steven. Just in case that was relevant to anyone else :-)
That's pretty much what you said then. Have you not returned to that mailing list because you no longer feel like part of the community?
I seem to remember saying at the time that if I'd received the message for Steven as it was stated in public, I would not show up on bended knee as a supplicant as it appeared to demand.
I did not intend meanness. I absolutely did intend rebuke; at the time Soni was one poorly worded post away from my killfile, and I had seriously considered reporting several previous posts for CoC violations. The only reason I didn't was that I was under the impression that the list admins had seen those posts and chosen not to react to them in public. To be honest, I probably wouldn't have cared about the language had Soni not been on such thin ice with me already. As Steven correctly picked up, "Sunshine", "Sunny Jim" and the like are unfriendly "polite" nicknames in British English usage. Sorry, but it didn't even occur to me that I needed to explain that. My bad. "Evening, sunshine," is the sort of thing you expect a police constable to see just ahead of "You're nicked." It does not denote any goodwill towards the subject whatsoever. Think of the US Deep South; when the "Bless your heart"s come out, you know the speaker is really angry.
Yes, definitely my bad. Sorry about that.
It was a rebuke. I felt (and still feel) that Soni's behaviour needed rebuking quite hard. This merely happened to be the point at which I could join the conversation. Compared to what I believe was deserved this was a mild rebuke, but it was definitely a rebuke. -- Rhodri James *-* Kynesim Ltd
Rhodri James wrote:
Please do not assume Titus and I read every post. We both have other obligations on top of administering this list, so assume we are too busy to read anything here and if something should come to our attention then please explicitly bring it to our attention.
On 19/02/2020 00:26, Steven D'Aprano wrote:
Actually we aren't all adults here, I happen to know, but never mind. I still thought Soni's language was inappropriate and just cautioning them was appropriate. If I thought "shitty wrapper" was unacceptable, I wouldn't have bothered cautioning them and just reported the CoC breach. -- Rhodri James *-* Kynesim Ltd
On 02/19/2020 08:55 AM, Soni L. wrote:
On 2020-02-18 5:33 p.m., Soni L. wrote:
Similar to len(). Just a [crappy] wrapper for __valid_getitem_requests__.
Correction: Just a simple wrapper for __valid_getitem_requests__.
Hopefully unsurprisingly, those words do not mean the same: simple: little if any extra functionality, possibly just a name redirect crappy: serious performance penalties, weird/incorrect edge cases, different order of parameters, hard to understand code, etc. -- ~Ethan~
On 2020-02-18 5:08 p.m., Rhodri James wrote:
Similar to len(). Just a shitty wrapper for __valid_getitem_requests__. We kinda have a requirement that __iter__ and __contains__ have to be related, see e.g. dict.__iter__ and dict.__contains__. In an ideal world dict would yield pairs by default. and dict() would just iterate whatever. But anyway I digress. As far as my proposal goes, __valid_getitem_requests__ would provide slightly different semantics for converting-to-dict: dict(items(['a', 'b', 'c'])) == {0: 'a', 1: 'b', 2: 'c'} dict(items({1: 'a', 2: 'b'})) == {1: 'a', 2: 'b'} # TODO find other things with __getitem__ and put them here In other words: it'd allow creating a dict whose __getitem__ behaviour is similar to that of the original collection, in the most straightforward case. (note how the first dict doesn't contain negative indices or the infinitely many slice objects that can be used to index a three-element list.) (perhaps a better name for this property is needed? how about "canonical keys"? as in, __valid_getitem_requests__ should contain only the canonical "keys" (the python datamodel specifically says __getitem__ takes a self and a key[1]) of a collection, and their respective values.) [1] https://docs.python.org/3/reference/datamodel.html#object.__getitem__
On 2020-02-18 5:43 p.m., Chris Angelico wrote:
Oh yes absolutely. I have a whole library focused on this sort of "canonical keys"-based iteration: https://ganarchy.autistic.space/project/0f74bd87a23b515b45da7e6f5d9cc8238044... (Friendly warning, it does handle sets in a rather unusual way. I've never used that feature outside unit tests, either. Pay no attention to that however, as it's not relevant to this proposal.)
On Wed, Feb 19, 2020 at 7:58 AM Soni L. <fakedme+py@gmail.com> wrote:
Too short, didn't read. Specifically, I'm not going to go digging through your links to find code to justify why you want this feature. Post some code here, or at very least, link directly to the code that would be improved by this feature. Soni, you're very close to my mental blocklist. Please put a bit more effort into your posts to make it clear what you're actually proposing, rather than making each of us do the work. ChrisA
On Tue, Feb 18, 2020 at 1:06 PM Chris Angelico <rosuav@gmail.com> wrote:
That link leads to a page where there's a link to https://soniex2.autistic.space/git-repos/abdl.git/ which is not accessible without further authentication.
I have to agree with Chris here -- Soni, please put a bit more effort in your posts, if you want to remain welcome here. From your URL I gather you may not have the best social skills. That's fine. We all try to accommodate a diverse audience here. But at some point, if you don't at least show that you're trying your best, you will wear out your welcome here. Maybe that has happened elsewhere. Don't blame it entirely on others or whatever handicap you may have (we have no idea here, except that we observe your posts to be somewhat cryptic). You have started a large number of threads in a short amount of time, and most of them have generated little of use to the Python community. Please think about how you can contribute something more constructive. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On 2020-02-18 6:04 p.m., Chris Angelico wrote:
The documentation states: "ABDL expressions are regex-like constructs for matching and validating object structures. They can be used with JSON and similar formats, and even self-referential data structures." "ABDL expressions have the ability to iterate, index, validate and filter data structures, through the use of the syntax elements below." "An arrow is ``->`` and indicates indexing/iteration (Mappings, Sequences, Sets). Whether indexing or iteration is used is defined by the elements that follow, with iteration being used by default." "A variable is a sequence of alphanumeric characters, not starting with a digit. A ``(key, value)`` tuple containing the respective matched element will be identified by this name in the results dict." And this sums up the core aspect of the library: the library is built on the idea of iterating "canonical keys" and their corresponding values, and validating and filtering them. ("[...] iteration being used by default", meaning indexing takes additional syntax - this is unlike some JSON and XML query languages where indexing takes /less/ syntax and filtering is generally highly limited and verbose.) Iteration is done thusly: def _pairs(obj): if isinstance(obj, collections.abc.Mapping): return iter(obj.items()) elif isinstance(obj, collections.abc.Sequence): return iter(enumerate(obj, 0)) elif isinstance(obj, collections.abc.Set): return iter(((e, e) for e in obj)) else: # maybe there's more stuff I can implement later raise TypeError I think having a proper protocol for this behaviour would be beneficial, as no (monkey)patching would be needed to add support for additional collections and data types. The comment saying there may be additional data types I've missed would then become irrelevant, as they'd be covered by __valid_getitem_requests__. Sometimes you just want to treat everything as a (read-only) dict. Mostly when dealing with user input (e.g. configs), but not only.
On Tue, Feb 18, 2020 at 1:45 PM Soni L. <fakedme+py@gmail.com> wrote:
But what will you do with the pairs? For mappings and sequences, the pairs represent (key, value) or (index, value) pairs, which makes sense, since (as you write in your first post) one can write obj[key] to obtain values from a mapping or sequence (key ~~ index). But then you go on and propose returning pairs (value, value) for sets, and there's nothing corresponding to obj[key]. So I'm not sure what your intent for the proposed is. Why do you want pairs for everything? Suppose obj is a number x, perhaps you would want to get a single pair (x, x) from the iteration? (I know that's probably bending the metaphor too far. :-) Even if you were to answer this question, consider that perhaps your application is rather unique in this insistence on pairs. Or perhaps you should've stopped at proposing an API that unifies mappings and sequences (which more sensible since both support __getitem__) -- isn't that what your original post was about? However even for mappings and sequences the analogy only goes so far, since *iterating* over them behaves very differently: for a mapping, iteration yields keys, while for a sequence, it yields values. A legitimate question that comes up from time to time is actually how to *distinguish* between mappings and sequences -- se the other python-ideas thread that I mentioned previously. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On 2020-02-18 7:03 p.m., Guido van Rossum wrote:
Yep, I'll bring you back to the previous post: "(Friendly warning, it does handle sets in a rather unusual way. I've never used that feature outside unit tests, either. Pay no attention to that however, as it's not relevant to this proposal.)" I kinda just really wish sets did easy interning with a very low chance of accidentally mapping two different things to the same (or otherwise wrong) objects (e.g. intern_cache["a"]=None), and so I decided to encode that into my _pairs. But that is unrelated to this proposal.
I should've stopped myself from adding this weird behaviour with sets. I've never even used it yet. Sequences and dicts, I do use, but not sets.
Yeah I can't say I don't have some "I like the way Lua does this" feelings. I'll be honest tho, if I were to implement fully separate behaviour for lists and dicts, the fact that dicts iterate by key and lists iterate by value means I would have to go with either a slower implementation (such as an operator that basically just converts an iterator of keys into one of values - or at least I'm pretty sure python doesn't optimize "indexing a dict with the key you're iterating", even if that may or may not be a cpython implementation detail) or one with loads of duplicated code (different code for lists and for dicts, and different arrow operators to select between them). I opted for this instead, it seemed like an acceptable tradeoff altho I'm not sure the python community likes it.
On 18/02/2020 20:33, Soni L. wrote:
Language, sunshine. Do you have a use case for this, or is it just theoretically nice to have? I have to say it isn't nice enough for me to actually want it, and I say that as someone who regularly forgets that iterating over a dict gets you its keys. -- Rhodri James *-* Kynesim Ltd
Presumably this was inspired by the other thread, "Add __keys__ or __items__ protocol", where an __items__() method was proposed instead of recognizing mappings by the presence of a keys() method. On Tue, Feb 18, 2020 at 12:46 PM Rhodri James <rhodri@kynesim.co.uk> wrote:
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...>
On Tue, 18 Feb 2020 at 20:47, Rhodri James <rhodri@kynesim.co.uk> wrote:
This looks to me like Lua's `items()` function. It's useful in Lua, but I'm not sure there's any obvious reason to assume it'll be a natural fit for idiomatic Python code. As people have said, do you have a good Python example of real code where this would be useful? (And where dct.items() or enumerate(lst) wouldn't be sufficient - code that expects to work equally with lists or dictionaries seems more Lua-like than Pythonic, IMO). Paul
On Tue, Feb 18, 2020 at 09:04:21PM +0000, Paul Moore wrote:
This looks to me like Lua's `items()` function.
I can't find that function listed anywhere. https://www.lua.org/manual/5.3/ Are you maybe thinking of `pairs()`? https://www.lua.org/manual/5.3/manual.html#pdf-pairs
In Lua, lists (arrays) *are* dictionaries, they keys are merely implicitly set to consecutive integers. My personal experience is that the analogy between (index, item) and (key, item) is more useful in theory and practice, but if I ever needed to do such a thing, a thin helper function: def items(dict_or_seq): try: return obj.items() except AttributeError: return enumerate(obj) would do the job. I don't think this is useful or important enough to turn it into a language-wide protocol. It's not as if the concept has much, if any, use outside of mappings and sequences. -- Steven
On Tue, Feb 18, 2020 at 08:44:07PM +0000, Rhodri James wrote:
Language, sunshine.
What about it? Just for the record, I had no issues with Soni's language, but I do object to attempts to shame him for it. This isn't the Disney Chanel, it's been over 100 years since Victoria was Queen of England, and we're all adults here. -- Steven
On 02/18/2020 04:26 PM, Steven D'Aprano wrote:
On Tue, Feb 18, 2020 at 08:44:07PM +0000, Rhodri James wrote:
Since when is simply drawing one's attention to something shaming?
Really? I didn't realize those under the legal age of majority were not allowed on this list. At any rate, we should be able to control what we say and how we say it, adult or not. -- ~Ethan~
On Feb 18, 2020, at 17:11, Ethan Furman <ethan@stoneleaf.us> wrote:
Isn’t the point of having a Code of Conduct that we shouldn’t have to have these arguments? Or, if there is an argument about whether a single figurative use as a word qualifies as “excessive swearing” per the CoC, surely an -ideas thread isn’t the place to have it?
Yeah, I very much doubt that "shitty wrapper" constitutes as excessive swearing. It'd be one thing if the post contained many or was directed at someone else, but the OP was using it to refer to something they personally wrote. Personally, I think it's a non-issue in that sort of context. On Tue, Feb 18, 2020 at 10:40 PM Andrew Barnert via Python-ideas < python-ideas@python.org> wrote:
I've been sitting on this email for the best part of a week. It hasn't been an easy decision to decide to send it, because I expect it will ruffle some feathers, including someone who, last time I ruffled his feathers, banned me from one of the other mailing lists. But I am part of this community, and if I'm too scared to state my opinions for fear of retaliation, then I'm not really part of the community, the supposed community values of being open and welcoming is a sham, and it's best if I find out now. When I posted my reply to Rhodi, I wasn't intending to shine the spotlight on his comment to Soni more than the simple statement I made. But since Ethan disagreed with me, I should elaborate. On Tue, Feb 18, 2020 at 05:09:39PM -0800, Ethan Furman wrote:
Of course drawing attention to something can often be shaming and a disguised attack: "You're fat. He's filthy. She's promiscuous. Your language is unacceptable and bad (and by implication, so are you)." Having attention drawn to your social transgression is shaming. That's why we do it: to stop people from transgressing again. "Language" is short for "Watch your language", a command, not an observation or even a suggestion. But I'm pretty sure that you, Ethan, already knew that. Defending Rhodi on the basis that he's merely stating a fact is disingenious. In Commonwealth English, calling someone you don't know "sunshine" is a mock-friendly but actually hostile act, rather like the way "pal" or "buddy" can be used in American English. Even if you did not know this, you surely knew that in the context of this email thread, calling someone "sunshine" is condescending, like calling them "pumpkin" or "sweety-pie". What Rhodi did was not "simply drawing one's attention", it was a rebuke. Ethan, I find it difficult to believe that you did not know that, and that your question was made in good faith.
Ethan, you've been involved with the Python community long enough to know the meaning of "we're all adults here", and it doesn't mean anything about legal age of majority. The point is, this is not a child-friendly space, this is an space for adults. We shouldn't be expected to dumb-down our conversation to that suitable for six year olds. If there are any underage people on this list, I expect them to (1) behave like adults, and (2) not freak out when the rest of us behave like adults. And that includes using "grown up words" (dear gods, why are we even having this conversation?) that I am pretty sure most of us, if not all of us, use off-list, hear our peers use, hear at the movies, on television, and on radio. Besides, I know what kids are like. I was one once, as we all were. The idea that an unsupervised child on the Internet doesn't already know this "bad word" by the time they're eight is at best laughably naive, if not patronising and demeaning. When the CoC was first debated, the PSF promised that the CoC would not be used to ban "rude words". Have they reneged on that promise? If not, Soni's language shouldn't even be an issue and Rhodi's rebuke was unnecessary. If they have, then we have a much more serious problem with this community.
At any rate, we should be able to control what we say and how we say it, adult or not.
Indeed we should. And if that means we want to call a shitty piece of software a shitty piece of software, especially when it's our own software, we shouldn't need fear that somebody will tell us off for it. -- Steven
On Mon, Feb 24, 2020 at 10:04 AM Steven D'Aprano <steve@pearwood.info> wrote:
For the record, I think Soni's original description (implying that the thin wrappers like iter(x) calling x.__iter__() etc are in some way bad) was rude, but not a CoC violation or anything. It's the sort of rude that colours someone's posts and thus other people's views of their arguments, but shouldn't get the person banned. But it WAS rude, and I don't think Rhodri was wrong to call that out. If you're going to rail on the code patterns that currently exist (profane language is mostly irrelevant here), that really doesn't do your arguments any favours, and it's fine IMO to call someone out for it just as much as you'd call someone out for posting their entire feature request in all-caps with no punctuation or line breaks. ChrisA
WHATS WRONG WITH TOP POSTING IN ALL CAPS AND NO PUNCTUATION!?!?!?! :) No one has complained to the moderators that CoC is being violated. and if they did, there’s a whole bunch of safeguards to ensure that it’s not being misused. I do occasionally (once or twice a year?) place specific members back on moderate temporarily so that I can intercept heated discussions before they spiral out of control, but I’ve tried to keep moderation rapid enough that I don’t think it’s intrusive. Nothing on this list is time critical, IMO, and I also note that new members are moderated _by default_. Also, if someone thinks that the CoC is being over applied (!?) or the list moderation is too heavy handed (!?) on this list, this list is also not the place to discuss it. That place would be conduct-wg@python.org, and I’m sure they would appreciate any concerns anyone has to convey. best, —titus
This is an honest question: Is it helpful to anyone taking part in this conversation about the conversation to have it? I don’t think it’s helpful to the list as a whole, honestly. I suggest next time the conversation gets a bit heated or offensive, and someone points it out, that everyone involved says “I’m sorry”, then takes a little break, and then the technical discussion can resume. One other suggestion is that if you are finding someone’s posts annoying or offensive, then either block them quietly or send them a direct note explaining why. Having that discussion in public just stirs things up for the rest of us. On the other hand, if folks really do find these conversations about the conversation helpful, maybe start a new thread? -CHB
-- Christopher Barker, PhD
Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On 02/23/2020 03:01 PM, Steven D'Aprano wrote:
The relevant posts from the thread in question: https://mail.python.org/pipermail/python-list/2018-September/887394.html https://mail.python.org/pipermail/python-list/2018-September/887398.html https://mail.python.org/pipermail/python-list/2018-September/887401.html https://mail.python.org/pipermail/python-list/2018-September/887445.html https://mail.python.org/pipermail/python-list/2018-September/887493.html https://mail.python.org/pipermail/python-list/2018-September/887418.html https://mail.python.org/pipermail/python-list/2018-September/887464.html
That's pretty much what you said then. Have you not returned to that mailing list because you no longer feel like part of the community?
When I posted my reply to Rhodri, I wasn't intending to shine the spotlight on his comment to Soni more than the simple statement I made:
(Rhodri has two 'r's in his name.)
But since Ethan disagreed with me, I should elaborate.
Rhodri James wrote: ------------------ Language, sunshine. Steven D'Aprano wrote: --------------------- What about it? Just for the record, I had no issues with Soni's language, but I do object to attempts to shame him for it. Ethan Furman wrote: ------------------ Since when is simply drawing one's attention to something shaming? Steven explains: ---------------
To me, "shaming" also requires a certain amount of meanness or cruelty (the "attack" part), often suggested by the tone or body language of the person commenting. Hopefully we are all aware that tone is hard to achieve in the written word, especially across continents and language barriers; body language is, of course, completely absent.
Had I thought about it that way I would agree with you. I did not, however, see his comment as a command, nor do I agree that "Language" is always short for "Watch your language", especially in the often terse environment of mailing lists.
No, I did not know that. And "pal" and "buddy" are no more hostile than any other American English word unless used in a certain way -- a way made explicit by tone and body language. In fact, when I first read Rhodri's post I thought it meant something along the lines of: programming language -> pie in the sky perfection. Hence, it is possible to be too terse.
I'm sorry you think so poorly of me. Steven previously said: ---------------------- This isn't the Disney Chanel, it's been over 100 years since Victoria was Queen of England, and we're all adults here. Ethan had replied: ----------------- Really? I didn't realize those under the legal age of majority were not allowed on this list.
Yes, it means we are responsible for our own code.
Using words that accurately describe a situation without using profanity is not "dumbing down".
If "behaving like adults" was the accepted norm, we wouldn't have a CoC (and No, I am NOT suggesting that the OP, or this post, are even remotely close to a CoC violation).
And that includes using "grown up words" (dear gods, why are we even having this conversation?)
Because you don't like people disagreeing with you about what is acceptable list behavior?
that I am pretty sure most of us, if not all of us, use off-list, hear our peers use, hear at the movies,on television, and on radio.
Which is irrelevant to what is expected on the Python lists.
It's a matter of courtesy. I, for one, greatly appreciate posters who take the time to formulate their arguments and/or problems without using emotionally charged words and/or expletives which usually add little and can easily take away from the intended conversation (such as your querying of the gods a few moments ago).
When the CoC was first debated, the PSF promised that the CoC would not be used to ban "rude words". Have they reneged on that promise?
I don't know -- maybe you should ask them? Or are you conflating Rhodri with the PSF?
If not, Soni's language shouldn't even be an issue and Rhodri's rebuke was unnecessary.
We'll have to agree to disagree on whether Rhodri's post was a rebuke or a reminder.
If they have, then we have a much more serious problem with this community.
Indeed. I would hate to think that every little disagreement needs to be moderated and judged by the PSF. Ethan wrote: ----------- At any rate, we should be able to control what we say and how we say it, adult or not.
Is this one of those language barrier thingies? Because to me, "telling us off for it" would be quite a bit harsher than the two words Rhodri used. -- ~Ethan~
On Mon, 24 Feb 2020 at 08:52, Ethan Furman <ethan@stoneleaf.us> wrote:
I feel that this discussion has got out of hand, and it would be good for everyone to take a breath and move on. However, *as a data point* I will say that I took Soni's original statement as being a mildly self-deprecating comment about a bit of code they had written (but hadn't, IIRC, actually quoted in the posting, just referred to), which in their view was not particularly clever but got the job done. I personally wouldn't have used the word "shitty" in an email, but I quite likely would have in spoken conversation, and I'm personally much more formal in email than when speaking, and I know many other people aren't. So I have no problem with Soni's use of the word. I took Rhodri's comment as a mild rebuke - worded as it was to be light-hearted and therefore not to be taken *too* seriously. Maybe he doesn't like the language himself, maybe he was enforcing a standard he felt was appropriate for the list, I don't know, and it doesn't really matter. His comment was brief, and full of cultural implications which (as an English person) I suspect I caught but which may have been missed by others who are from different backgrounds. Maybe it was a mistake being quite so colloquial, but it's an easy one to make if you're not spending inordinate amounts of time fine-tuning your emails. Steven's objection to "shaming", while maybe technically accurate, seemed to ignore a lot of the nuance. Maybe that's down to personality, or cultural differences. But I do think it failed to assume (or maybe "allow for the possibility of" would be more accurate) good intentions. But I'm fine with assuming that it was itself a well-intentioned attempt to caution people against giving offense, which didn't have the intended effect. Hopefully, if I've misinterpreted anyone's actual meaning in the above, I've erred on the side of assuming good intention. If anyone feels otherwise, I apologise. I suggest that any harm that might have been done by the original post is getting lost now in the increasing heat of the argument, and we'd be better if we simply let the debate drop at this point. Paul
As the instigator here, I think it's my duty to clarify my intent on a few things. On 24/02/2020 08:49, Ethan Furman wrote:
I have no problems at all with you stating your mind about this, Steven. Just in case that was relevant to anyone else :-)
That's pretty much what you said then. Have you not returned to that mailing list because you no longer feel like part of the community?
I seem to remember saying at the time that if I'd received the message for Steven as it was stated in public, I would not show up on bended knee as a supplicant as it appeared to demand.
I did not intend meanness. I absolutely did intend rebuke; at the time Soni was one poorly worded post away from my killfile, and I had seriously considered reporting several previous posts for CoC violations. The only reason I didn't was that I was under the impression that the list admins had seen those posts and chosen not to react to them in public. To be honest, I probably wouldn't have cared about the language had Soni not been on such thin ice with me already. As Steven correctly picked up, "Sunshine", "Sunny Jim" and the like are unfriendly "polite" nicknames in British English usage. Sorry, but it didn't even occur to me that I needed to explain that. My bad. "Evening, sunshine," is the sort of thing you expect a police constable to see just ahead of "You're nicked." It does not denote any goodwill towards the subject whatsoever. Think of the US Deep South; when the "Bless your heart"s come out, you know the speaker is really angry.
Yes, definitely my bad. Sorry about that.
It was a rebuke. I felt (and still feel) that Soni's behaviour needed rebuking quite hard. This merely happened to be the point at which I could join the conversation. Compared to what I believe was deserved this was a mild rebuke, but it was definitely a rebuke. -- Rhodri James *-* Kynesim Ltd
Rhodri James wrote:
Please do not assume Titus and I read every post. We both have other obligations on top of administering this list, so assume we are too busy to read anything here and if something should come to our attention then please explicitly bring it to our attention.
On 19/02/2020 00:26, Steven D'Aprano wrote:
Actually we aren't all adults here, I happen to know, but never mind. I still thought Soni's language was inappropriate and just cautioning them was appropriate. If I thought "shitty wrapper" was unacceptable, I wouldn't have bothered cautioning them and just reported the CoC breach. -- Rhodri James *-* Kynesim Ltd
On 02/19/2020 08:55 AM, Soni L. wrote:
On 2020-02-18 5:33 p.m., Soni L. wrote:
Similar to len(). Just a [crappy] wrapper for __valid_getitem_requests__.
Correction: Just a simple wrapper for __valid_getitem_requests__.
Hopefully unsurprisingly, those words do not mean the same: simple: little if any extra functionality, possibly just a name redirect crappy: serious performance penalties, weird/incorrect edge cases, different order of parameters, hard to understand code, etc. -- ~Ethan~
participants (14)
-
Andrew Barnert -
Antoine Pitrou -
Brett Cannon -
C. Titus Brown -
Chris Angelico -
Christopher Barker -
Ethan Furman -
Guido van Rossum -
Kyle Stanley -
MRAB -
Paul Moore -
Rhodri James -
Soni L. -
Steven D'Aprano