Add "default" kwarg to list.pop()
Sorry in advance if this has been proposed in the past but I couldn't find anything on python-ideas:
FWIW my use case consists in reading entries from /proc/diskstats where lines can have a variable number of fields depending on the kernel version: https://github.com/giampaolo/psutil/issues/1354#issuecomment-434495870 https://github.com/giampaolo/psutil/blob/d8b05151e65f9348aff9b58da977abd8cac... As such it would be convenient to set missing fields to 0 as "reads = fields.pop(default=0)" instead of catching IndexError every time. Extra: for consistency set.pop() should probably have the same. Thoughts? -- Giampaolo - http://grodola.blogspot.com
On Wed, Oct 31, 2018 at 02:25:25AM +0200, Serhiy Storchaka wrote:
It is just
l.pop() if l else default
It might *do* the same thing, but it doesn't communicate the programmer's intention as well. {}.pop('key', default) could be written using LBYL too, but the intention is much clearer given an explicit default argument. The only advantage of the "if l" version is that if the default is expensive to calculate, we can short-circuit it.
or
(l or [default]).pop()
That's clever, but it is also wasteful, building a single-item list only to immediately pop the item out of it and throw the list away. [steve@ando ~]$ python3.5 -m timeit -s "l = []" "l.pop() if l else None" 10000000 loops, best of 3: 0.0739 usec per loop [steve@ando ~]$ python3.5 -m timeit -s "l = []" "(l or [None]).pop()" 1000000 loops, best of 3: 0.421 usec per loop -- Steve
As a user I always found a bit disurbing that dict pop method have a default while list and set doesn't. While it is way more computationally easy to check wether a list or a set is empty that to check if a key is in a dict, it still create a signature difference for no real reason (having a default to a built-in in python is pretty standard). It would be nice if every built-in/method of built-in type that returns a value and raise in some case have access to a default instead of raise, and not having to check the doc to see if it supports a default. We could for exemple ask ourselves wether or not list.index should have a default, as it is a method that we explecitely excpect to return a value and might just raise instead. 2018-10-31 2:08 GMT+01:00 Steven D'Aprano <steve@pearwood.info>:
-- -- *Nicolas Rolin* | Data Scientist + 33 631992617 - nicolas.rolin@tiime.fr <prenom.nom@tiime.fr> *15 rue Auber, **75009 Paris* *www.tiime.fr <http://www.tiime.fr>*
I think l.pop() if l else None is good enough. I think it's pretty obvious that a developer means "Pop the list if the list is not empty, and return None if the list is empty ". - Ron [image: Facebook] <http://www.facebook.com/ron.reiter> [image: Twitter] <http://twitter.com/#!/ronreiter> [image: LinkedIn] <https://il.linkedin.com/in/ronreiter> On Wed, Oct 31, 2018 at 11:24 AM Nicolas Rolin <nicolas.rolin@tiime.fr> wrote:
I think the same way about set.pop, list.pop. About .index I agree adding default= would make sense but that's not exactly the same thing as the others. Do we have somewhere else a place where a linear search already accepts a default= kwarg ? def index(self, x): for i,y in enumerate(self): if x == y: return i raise ValueError We do have "next", the default version of .index is currently implentable as : next((i for i, y in enumerate(self) if x == y), -1) If I want -1 for default. Le mer. 31 oct. 2018 à 10:23, Nicolas Rolin <nicolas.rolin@tiime.fr> a écrit :
On Wed, Oct 31, 2018 at 8:24 PM Nicolas Rolin <nicolas.rolin@tiime.fr> wrote:
https://www.python.org/dev/peps/pep-0463/ wants to say hi. ChrisA
On Wed, Oct 31, 2018 at 08:41:28PM +1100, Chris Angelico wrote:
PEP 463 is too busy crying bitter tears in the corner after being rejected to say "Hi". I don't think this is the same thing: PEP 464 is about being able to catch arbitrary exceptions in arbitrary expressions in an ad-hoc manner. Nicholas' suggestion is about making a consistent strategy of avoiding the need to catch exceptions. I don't think I would agree with a broad rule "anything that raises can return a default value" -- I don't think it makes sense to have, let's say, len(obj, default=2). But on a case-by-case basis, it works for me. If this were Ruby, and we could monkey-patch the built-in types, my reasoning would go something like this: - if I have to write "L.pop() if L else default", I should write it in place; - the third time I write it in any one module, I ought to factor it out into a helper function; - the third time I write the helper function, I ought to just monkey-patch the built-in and be done with it. I haven't been counting, but I'm pretty sure I've written "get(L, index, default)" and "pop(L, index, default)" helpers at least three times now. Possibly even three times each :-) In Python, of course, we can't add methods to built-ins. But I think this functionality makes good sense, especially once we remember that pop takes an optional index. Spot the bug in this: L.pop(idx) if len(L) > idx else default -- Steve
On Wed, Oct 31, 2018 at 9:14 PM Steven D'Aprano <steve@pearwood.info> wrote:
And that's exactly why a broad rule of "anything that raises can be wrapped in a catcher" does make sense. Hence it may not be the same thing, but it is an alternative solution that doesn't require specifically angling for consistency. ChrisA
On Wed, Oct 31, 2018 at 09:31:37PM +1100, Chris Angelico wrote:
True. I'm not arguing against the earlier PEP, I was in favour of it too. But even if we had exception-catching expressions, it would still make sense to add a default value to list.pop and perhaps a list.get method too. -- Steve
On Wed, Oct 31, 2018 at 2:42 AM Chris Angelico <rosuav@gmail.com> wrote:
https://www.python.org/dev/peps/pep-0463/ wants to say hi.
That was exactly my reaction, too, and usually is whenever one of these "add a default" or similar ideas pops up. 463 should be re-examined, I was very hopeful when it went through the wringer the first time, but alas it was not to be.
Oooh, PEP463, you're reason with I switch to LBYL or write studpid try except functions so much times. Oh and also the local assignement "let/where/statement" :D x = (y+1 where y = 3.14) because x = [y+1 for y in [3.14]][0] is an overkill and ugly. Should I write a PEP even though I know it's going to be rejected because the mailing list was not really into it ? Le mer. 31 oct. 2018 à 18:37, Michael Selik <mike@selik.org> a écrit :
On Thu, Nov 1, 2018 at 6:25 AM Robert Vanden Eynde <robertve92@gmail.com> wrote:
There's nothing wrong with writing PEPs that have a low likelihood of being accepted. In fact, some PEPs have specifically been written with the purpose of rejecting them - creating a sort of FAQ document ("this has been proposed many times, here's the arguments in favour of it, and it's rejected because X and Y"). That said, though, you may well not need to go to that effort. What is being asked for here (if I'm not misreading) is a relatively simple enhancement to a method on a built-in type (or a small handful of types). If that garners reasonable support, the next step wouldn't be a PEP, it'd just go straight to a tracker issue and a pull request. For myself, I'm +0 on adding default=. It'd be a minor convenience on a very small number of cases. But it wouldn't bother me at all so it wouldn't be a problem if it were to land. ChrisA
On Wed, Oct 31, 2018 at 12:31 PM Chris Angelico <rosuav@gmail.com> wrote:
If the consensus is to add a default keyword parameter for the rest of the get/pop methods on built-in types, it'd be reasonable to write an addendum to PEP 463 that mentions what is being established as the Pythonic interface: When a method can raise IndexError or KeyError, the method should provide an optional default which suppresses the error.
So, not all modification come from a PEP ? Where are the functionalities discussed for future python version ? Only on the list ? Is there a place where people or big projects can vote for functionalities ? Recently I saw on the 3.8 changelog "added math.dist(a,b)" and saw the discussion on the bug tracker. The last comment was "I don't understand why this was added, as the discussion was more in favor of supporting *args for math.hypot". Who selects which PR are merged ?
On Thu, Nov 1, 2018 at 7:46 AM Robert Vanden Eynde <robertve92@gmail.com> wrote:
Correct. Some more info here: https://www.python.org/dev/peps/pep-0001/#what-is-a-pep
Where are the functionalities discussed for future python version ? Only on the list ?
A number of places. This list is for all manner of discussions, but doesn't have any authority to make actual decisions. If a proposal has decent support here, it will usually be taken then to python-dev and then the tracker, or sometimes straight to the tracker (if it's considered a trivial change).
Is there a place where people or big projects can vote for functionalities ?
No. Voting is not a thing in Python. That might change now that Guido has stepped down, but even if it does, it isn't likely to be where big projects get to vote - it'll be a matter of the governing body voting on the change. What you can do is *express support* for a proposal, giving your reasons for such support, as a means of convincing the core devs to accept the plan. (Offering to write the code can remove a barrier to acceptance, but that's not the only barrier.)
Who selects which PR are merged ?
The core devs. Exactly how a team of core devs can make unified decisions is a little up in the air at the moment, but broadly speaking, it's up to the people who have the power to push code to the main repository. Which, in turn, means that it's the core devs that you ultimately need to convince if you have something you want changed. Convincing people here on this list has no intrinsic meaning, but it's a great way to hash out the proposal, the objections to the proposal, and the counter-arguments, etc, etc. Which is pretty important :) ChrisA
Just English Vocabulary, what do you mean by "being in the air at the moment" ? Like, that's a subject that a lot of people in here like to talk ? Yes, to merge or not to merge, but people can UpVote/DownVote can't they ? :D Le ven. 2 nov. 2018 à 01:15, Chris Angelico <rosuav@gmail.com> a écrit :
On Fri, Nov 2, 2018 at 11:19 AM Robert Vanden Eynde <robertve92@gmail.com> wrote:
Just English Vocabulary, what do you mean by "being in the air at the moment" ? Like, that's a subject that a lot of people in here like to talk ?
"Up in the air" means uncertain, subject to change. https://idioms.thefreedictionary.com/up+in+the+air In this case, the governance model for the Python language is being discussed.
Yes, to merge or not to merge, but people can UpVote/DownVote can't they ? :D
Upvotes and downvotes don't mean anything. So, sure! It's like upvoting or downvoting one of your country's laws... nobody's stopping you (at least, I hope you live in a country where you're allowed to express your likes and dislikes), but it doesn't change anything unless you're one of the handful of people who actually make the decision. ChrisA
In this case, the governance model for the Python language is being discussed.
This was the info I was missing, where is it discussed ? Not only on this list I assume ^^
Upvotes and downvotes don't mean anything. [...]
Yes, that's why random people wouldn't vote. But like, voting between like the 10 core devs where they all have the same importance, that does help for choosing "to merge or not to merge", isn't it ?
On Fri, Nov 2, 2018 at 11:26 AM Robert Vanden Eynde <robertve92@gmail.com> wrote:
In this case, the governance model for the Python language is being discussed.
This was the info I was missing, where is it discussed ? Not only on this list I assume ^^
There are a number of PEPs in the 8000s that would be worth reading.
That is just one of the possible options - that decisions are made by vote. ChrisA
On Fri, Nov 02, 2018 at 10:48:22AM +0200, Serhiy Storchaka wrote:
I disagree that "the mailing list was not really into it". So far, I count 12 people who responded to the original post by Giampaolo. By my count, I see: * five people in favour; * three people against, or see no need for it; * four people I can't tell if they are for or against, (possibly neutral?) [1] I know that adding features isn't decided by majority vote, but it seems clear to me that there is a substantial set of Python users, perhaps a majority, who would find this feature useful and more obvious than the alternatives. [Serhiy]
It is better to not do this. PEP 572 was initially written with the intention to be rejected.
Sounds like an excellent reason to write a PEP :-) There are some issues that ought to be addressed: - The status quo is easy to get wrong: # I've written this. More than once. L.pop(idx) if idx < len(L) else default is wrong if there is any chance of idx being negative. - The more common case of popping from the front of the list is not thread-safe: L.pop() if L else default - This clever trick is probably thread-safe (I think...) but it is wasteful and inefficient: (L or [default]).pop() and it isn't obvious how to adapt it efficiently if you need to pop from an arbitrary index. I came up with this: (L[idx:idx+1] or [default]).pop() but it is doubly wrong. - The obvious thread-safe EAFP idiom is a try...except statement, so it needs to be wrapped in a helper function to use it in expressions. That adds more overhead. The proposed .get(idx, default=x) and .pop(idx, default=x) signatures ought to be obvious and unsurprising to any moderately experienced Python programmer. These aren't complicated APIs. On the other hand: - I'm not volunteering to do the work (I don't know enough C to write a patch). Unless somebody has a patch, we can't expect the core devs who aren't interested in this feature to write it. (Hence, status quo wins a stalemate.) [1] "What makes a man turn neutral? Lust for gold? Power? Or were you just born with a heart full of neutrality?" -- Captain Zapp Brannigan -- Steve
On 11/2/18 7:39 AM, Anders Hovmöller wrote:> little improvement. I'm having a hard time seeing a real use case. Giampaolo's original post contains this link: https://github.com/giampaolo/psutil/blob/d8b05151e65f9348aff9b58da977abd8cac... Yuck (from an aesthetics standpoint, not a functional standpoint). :-) There's an impedance mismatch between the data, which is structured and has changed apparently arbitrarily between Linux releases, and the return value of string.split, which is an ordered collection. This code effectively hides that mismatch and yields Python tuples, which represent structured data. I can certainly see the desire for a simpler solution (for some definition of simpler), but how would adding a default parameter to list.pop make this code any simpler? Dan
On Fri, Nov 2, 2018 at 3:59 AM, Steven D'Aprano <steve@pearwood.info> wrote:
Well, it would be frustrating to have a feature accepted but not implemented, but the steps are separate. And it wouldn't have to be a core dev that implements it -- anyone with the C chops (not me :-) ) could do it. As an example, a good chunk of PEP 485 was implemented by someone else (I wrote the first prototype, but it was not complete), who I'm pretty sure is not a core dev. A core dev has to actually merge it, of course, and that is a bottleneck, but not a show stopper. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
I don't understand the rationale behind PEP 463's rejection. Guido says, "I disagree with the position that EAFP is better than LBYL, or "generally recommended" by Python. (Where do you get that?..."; but it's been in the official Python.org docs <https://docs.python.org/3/glossary.html> for a while and even provides a pretty good justification for why EAFP is preferable to LBYL (aside from the language calling EAFP "common", "clean", and "fast" that's notably absent from LBYL's description): "In a multi-threaded environment, the LBYL approach can risk introducing a race condition between “the looking” and “the leaping”. For example, the code, if key in mapping: returnmapping[key] can fail if another thread removes *key* from *mapping* after the test, but before the lookup. This issue can be solved with locks or by using the EAFP approach." Which brings me to the question: What happens when a PEP gets rejected? Is it final? Is there a process for reviving a PEP? I personally would love to have *both* more consistent methods on built-in classes AND exception handling expressions. I think the colon (and maybe 'except' keyword) could be replaced with an exclamation point: value = lst[2] except IndexError! "No value" or just: value = lst[2] IndexError! "No value" if that appeases the people who dislike the over-use of colons. A full exception list would have to be in parentheses which get's ugly, but would also be (I would wager) a less common form: dirlist.append(os.getcwd() (AttributeError, OSError as e)! os.curdir) That might need some work. I don't know if it's compatible w/ the compiler. It may have to start with "try" or something, but it seems pretty close to a workable solution. On Wed, Oct 31, 2018 at 4:42 AM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, Nov 7, 2018 at 6:04 AM Abe Dillon <abedillon@gmail.com> wrote:
Which brings me to the question: What happens when a PEP gets rejected? Is it final? Is there a process for reviving a PEP?
It remains as a permanent document. No, that isn't final; and the process for reviving a PEP basically consists of answering the objections that led to its rejection. There have been a few cases where a proposal lies dormant for years before finally being accepted (such as the matrix multiplication operator). So if you want to do that, open a new thread, and specifically respond to the issues in the PEP - anything named as a reason for rejection, and anything else that you think ought to be improved. ChrisA
On Tue, Nov 06, 2018 at 01:03:54PM -0600, Abe Dillon wrote:
I can't comment on Guido's question about "generally recommended", but as for the first part, I agree: neither EAFP nor LBYL is "better", they are both appropriate under different circumstances. Sometimes one is clearer and more efficient than the other. The only time I would say that EAFP is clearly better is when LBYL introduces "Time Of Check To Time Of Use" bugs.
Which brings me to the question: What happens when a PEP gets rejected? Is it final? Is there a process for reviving a PEP?
Nothing is final-final. You can try opening a competing PEP, or take over as champion of the existing PEP (assuming Chris is willing to step aside). You ought to respond to the reasons given in the rejection. It's probably a good idea to gauge the chances of success by asking on Python-Ideas and Python-Dev first, to avoid the core devs saying "Oh give it up, it's not going to happen!" after you've wasted time trying to revise a rejected PEP. [...]
And I think that this is precisely the sort of syntax that prompted Guido to write many years ago that language design is not merely a problem-solving exercise. Aesthetics are important. This is not just a matter of finding an unused character or two and hammering it into the the language. That's how you get Perl, which is not a pretty language.
Seeing that syntax, the phrase that came to my mind was not so much "close to workable" and more "kill it with fire!". -- Steve
neither EAFP nor LBYL is "better", they are both appropriate under different circumstances. Sometimes one is
clearer and more efficient than the other.
One of the reasons LBYL is sometimes cleaner than EAPF is because it has more support from the language in the form of an expression which is what PEP 463 intends to change. The only time I would say that EAFP is clearly better is when LBYL
introduces "Time Of Check To Time Of Use" bugs.
It also puts the intent of the logic up front instead of requiring the reader to scroll through a preamble of edge-case checks to get to what the code is actually trying to do. I think that this is precisely the sort of syntax that prompted
Guido to write many years ago that language design is not merely a problem-solving exercise.
The sort of syntax that prompted that post was "precisely" multi-line lambdas. Guido explained that he tried to throw people off the idea of multi-line lambdas by posing it as an unsolvable puzzle (which people promptly solved) when really he just thought the concept of multi-line lambdas was flawed to begin with. I agree with him on that point. The whole point of a lambda is that, in certain cases, they allow you to write more expressive code by saying in-line exactly what you want to do. It only works if that action is easily expressed in a line: button.onClick(lambda: print("Hello!")) If it's a long an complicated bit of code, the expressiveness of lambda is lost and it makes more sense to give it a name and write: button.onClick(doThatComplicatedThing) I'm not trying to solve a puzzle that implements an anti-pattern (unless you have some argument for why expressionized try-except would be an anti-pattern). Aesthetics are important. This is not just a matter of finding an unused
character or two and hammering it into the the language.
Yeah. That's why I didn't just try to find an unused character and hammer it into the language without paying any regard to aesthetics. I find value = lst[2] except IndexError! "No value" to be pretty well in keeping w/ Python's aesthetics because raising an exception pretty naturally fits with an exclamation point, but of-course; aesthetics are subjective. I know what Perl is BTW, and share your distaste for it. Seeing that syntax, the phrase that came to my mind was not so much
"close to workable" and more "kill it with fire!".
Funny, that's exactly how I felt about the None-aware operators, only; I didn't reject the entire concept simply because I disliked the syntax choice. I simply rejected the syntax choice because I disliked the syntax choice... On Tue, Nov 6, 2018 at 5:21 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Nov 01, 2018 at 09:36:19PM +0200, Serhiy Storchaka wrote:
That's not an expression, so there are limits to where and when you can use it. What we need is a helper function that wraps that, called "pop". And since this seems to be reoccuring request going back nearly 20 years now: https://mail.python.org/pipermail/python-dev/1999-July/000550.html https://stackoverflow.com/questions/31216428/python-pop-from-empty-list as is the more general get(list, index, default=None) helper: https://stackoverflow.com/questions/2574636/getting-a-default-value-on-index... https://stackoverflow.com/questions/2492087/how-to-get-the-nth-element-of-a-... https://stackoverflow.com/questions/5125619/why-doesnt-list-have-safe-get-me... https://stackoverflow.com/questions/17721748/default-value-for-out-of-bounds... we could save people from having to re-invent the wheel over and over again by add them to a new module called "things_that_should_be_list_methods_but_arent.py" *wink* -- Steve
31.10.18 13:08, Antoine Pitrou пише:
list.pop() corresponds two dict methods. With argument it corresponds dict.pop(). But there are differences: dict.pop() called repeatedly with the same key will raise an error (or return the default), while list.pop() will likely return other item. Without argument it corresponds dict.popitem() which doesn't have an optional default.
Does it make sense to draw some sort of parallel between next(myiterator, default="whatever") and mylist.pop(default="whatever")? They exhaust the iterator/list then start emitting the default argument (if provided). On Thu, Nov 1, 2018 at 2:46 PM Serhiy Storchaka <storchaka@gmail.com> wrote:
Yep that's what I just did in my previous mail. """ I think the same way about set.pop, list.pop. About .index I agree adding default= would make sense but that's not exactly the same thing as the others. """ Being picky: "TypeError: next() takes no keyword arguments", that's next(myierator, "whatever") ;)
On Fri, Nov 2, 2018 at 4:45 AM Serhiy Storchaka <storchaka@gmail.com> wrote:
I think there is one more important difference between dict and list. dict has .get(key[, default]), but list doesn't have it. If we add only `list.pop([default])`, it is tempting that using it even when they don't have to remove the item. Unnecessary destructive change is bad. It reduces code readability, and it may create hard bug. If this proposal is adding `list.get([index[, default]])` too, I still -0. I don't know how often it is useful. Regards, -- INADA Naoki <songofacandy@gmail.com>
On Wed, Oct 31, 2018 at 02:25:25AM +0200, Serhiy Storchaka wrote:
It is just
l.pop() if l else default
It might *do* the same thing, but it doesn't communicate the programmer's intention as well. {}.pop('key', default) could be written using LBYL too, but the intention is much clearer given an explicit default argument. The only advantage of the "if l" version is that if the default is expensive to calculate, we can short-circuit it.
or
(l or [default]).pop()
That's clever, but it is also wasteful, building a single-item list only to immediately pop the item out of it and throw the list away. [steve@ando ~]$ python3.5 -m timeit -s "l = []" "l.pop() if l else None" 10000000 loops, best of 3: 0.0739 usec per loop [steve@ando ~]$ python3.5 -m timeit -s "l = []" "(l or [None]).pop()" 1000000 loops, best of 3: 0.421 usec per loop -- Steve
As a user I always found a bit disurbing that dict pop method have a default while list and set doesn't. While it is way more computationally easy to check wether a list or a set is empty that to check if a key is in a dict, it still create a signature difference for no real reason (having a default to a built-in in python is pretty standard). It would be nice if every built-in/method of built-in type that returns a value and raise in some case have access to a default instead of raise, and not having to check the doc to see if it supports a default. We could for exemple ask ourselves wether or not list.index should have a default, as it is a method that we explecitely excpect to return a value and might just raise instead. 2018-10-31 2:08 GMT+01:00 Steven D'Aprano <steve@pearwood.info>:
-- -- *Nicolas Rolin* | Data Scientist + 33 631992617 - nicolas.rolin@tiime.fr <prenom.nom@tiime.fr> *15 rue Auber, **75009 Paris* *www.tiime.fr <http://www.tiime.fr>*
I think l.pop() if l else None is good enough. I think it's pretty obvious that a developer means "Pop the list if the list is not empty, and return None if the list is empty ". - Ron [image: Facebook] <http://www.facebook.com/ron.reiter> [image: Twitter] <http://twitter.com/#!/ronreiter> [image: LinkedIn] <https://il.linkedin.com/in/ronreiter> On Wed, Oct 31, 2018 at 11:24 AM Nicolas Rolin <nicolas.rolin@tiime.fr> wrote:
I think the same way about set.pop, list.pop. About .index I agree adding default= would make sense but that's not exactly the same thing as the others. Do we have somewhere else a place where a linear search already accepts a default= kwarg ? def index(self, x): for i,y in enumerate(self): if x == y: return i raise ValueError We do have "next", the default version of .index is currently implentable as : next((i for i, y in enumerate(self) if x == y), -1) If I want -1 for default. Le mer. 31 oct. 2018 à 10:23, Nicolas Rolin <nicolas.rolin@tiime.fr> a écrit :
On Wed, Oct 31, 2018 at 8:24 PM Nicolas Rolin <nicolas.rolin@tiime.fr> wrote:
https://www.python.org/dev/peps/pep-0463/ wants to say hi. ChrisA
On Wed, Oct 31, 2018 at 08:41:28PM +1100, Chris Angelico wrote:
PEP 463 is too busy crying bitter tears in the corner after being rejected to say "Hi". I don't think this is the same thing: PEP 464 is about being able to catch arbitrary exceptions in arbitrary expressions in an ad-hoc manner. Nicholas' suggestion is about making a consistent strategy of avoiding the need to catch exceptions. I don't think I would agree with a broad rule "anything that raises can return a default value" -- I don't think it makes sense to have, let's say, len(obj, default=2). But on a case-by-case basis, it works for me. If this were Ruby, and we could monkey-patch the built-in types, my reasoning would go something like this: - if I have to write "L.pop() if L else default", I should write it in place; - the third time I write it in any one module, I ought to factor it out into a helper function; - the third time I write the helper function, I ought to just monkey-patch the built-in and be done with it. I haven't been counting, but I'm pretty sure I've written "get(L, index, default)" and "pop(L, index, default)" helpers at least three times now. Possibly even three times each :-) In Python, of course, we can't add methods to built-ins. But I think this functionality makes good sense, especially once we remember that pop takes an optional index. Spot the bug in this: L.pop(idx) if len(L) > idx else default -- Steve
On Wed, Oct 31, 2018 at 9:14 PM Steven D'Aprano <steve@pearwood.info> wrote:
And that's exactly why a broad rule of "anything that raises can be wrapped in a catcher" does make sense. Hence it may not be the same thing, but it is an alternative solution that doesn't require specifically angling for consistency. ChrisA
On Wed, Oct 31, 2018 at 09:31:37PM +1100, Chris Angelico wrote:
True. I'm not arguing against the earlier PEP, I was in favour of it too. But even if we had exception-catching expressions, it would still make sense to add a default value to list.pop and perhaps a list.get method too. -- Steve
On Wed, Oct 31, 2018 at 2:42 AM Chris Angelico <rosuav@gmail.com> wrote:
https://www.python.org/dev/peps/pep-0463/ wants to say hi.
That was exactly my reaction, too, and usually is whenever one of these "add a default" or similar ideas pops up. 463 should be re-examined, I was very hopeful when it went through the wringer the first time, but alas it was not to be.
Oooh, PEP463, you're reason with I switch to LBYL or write studpid try except functions so much times. Oh and also the local assignement "let/where/statement" :D x = (y+1 where y = 3.14) because x = [y+1 for y in [3.14]][0] is an overkill and ugly. Should I write a PEP even though I know it's going to be rejected because the mailing list was not really into it ? Le mer. 31 oct. 2018 à 18:37, Michael Selik <mike@selik.org> a écrit :
On Thu, Nov 1, 2018 at 6:25 AM Robert Vanden Eynde <robertve92@gmail.com> wrote:
There's nothing wrong with writing PEPs that have a low likelihood of being accepted. In fact, some PEPs have specifically been written with the purpose of rejecting them - creating a sort of FAQ document ("this has been proposed many times, here's the arguments in favour of it, and it's rejected because X and Y"). That said, though, you may well not need to go to that effort. What is being asked for here (if I'm not misreading) is a relatively simple enhancement to a method on a built-in type (or a small handful of types). If that garners reasonable support, the next step wouldn't be a PEP, it'd just go straight to a tracker issue and a pull request. For myself, I'm +0 on adding default=. It'd be a minor convenience on a very small number of cases. But it wouldn't bother me at all so it wouldn't be a problem if it were to land. ChrisA
On Wed, Oct 31, 2018 at 12:31 PM Chris Angelico <rosuav@gmail.com> wrote:
If the consensus is to add a default keyword parameter for the rest of the get/pop methods on built-in types, it'd be reasonable to write an addendum to PEP 463 that mentions what is being established as the Pythonic interface: When a method can raise IndexError or KeyError, the method should provide an optional default which suppresses the error.
So, not all modification come from a PEP ? Where are the functionalities discussed for future python version ? Only on the list ? Is there a place where people or big projects can vote for functionalities ? Recently I saw on the 3.8 changelog "added math.dist(a,b)" and saw the discussion on the bug tracker. The last comment was "I don't understand why this was added, as the discussion was more in favor of supporting *args for math.hypot". Who selects which PR are merged ?
On Thu, Nov 1, 2018 at 7:46 AM Robert Vanden Eynde <robertve92@gmail.com> wrote:
Correct. Some more info here: https://www.python.org/dev/peps/pep-0001/#what-is-a-pep
Where are the functionalities discussed for future python version ? Only on the list ?
A number of places. This list is for all manner of discussions, but doesn't have any authority to make actual decisions. If a proposal has decent support here, it will usually be taken then to python-dev and then the tracker, or sometimes straight to the tracker (if it's considered a trivial change).
Is there a place where people or big projects can vote for functionalities ?
No. Voting is not a thing in Python. That might change now that Guido has stepped down, but even if it does, it isn't likely to be where big projects get to vote - it'll be a matter of the governing body voting on the change. What you can do is *express support* for a proposal, giving your reasons for such support, as a means of convincing the core devs to accept the plan. (Offering to write the code can remove a barrier to acceptance, but that's not the only barrier.)
Who selects which PR are merged ?
The core devs. Exactly how a team of core devs can make unified decisions is a little up in the air at the moment, but broadly speaking, it's up to the people who have the power to push code to the main repository. Which, in turn, means that it's the core devs that you ultimately need to convince if you have something you want changed. Convincing people here on this list has no intrinsic meaning, but it's a great way to hash out the proposal, the objections to the proposal, and the counter-arguments, etc, etc. Which is pretty important :) ChrisA
Just English Vocabulary, what do you mean by "being in the air at the moment" ? Like, that's a subject that a lot of people in here like to talk ? Yes, to merge or not to merge, but people can UpVote/DownVote can't they ? :D Le ven. 2 nov. 2018 à 01:15, Chris Angelico <rosuav@gmail.com> a écrit :
On Fri, Nov 2, 2018 at 11:19 AM Robert Vanden Eynde <robertve92@gmail.com> wrote:
Just English Vocabulary, what do you mean by "being in the air at the moment" ? Like, that's a subject that a lot of people in here like to talk ?
"Up in the air" means uncertain, subject to change. https://idioms.thefreedictionary.com/up+in+the+air In this case, the governance model for the Python language is being discussed.
Yes, to merge or not to merge, but people can UpVote/DownVote can't they ? :D
Upvotes and downvotes don't mean anything. So, sure! It's like upvoting or downvoting one of your country's laws... nobody's stopping you (at least, I hope you live in a country where you're allowed to express your likes and dislikes), but it doesn't change anything unless you're one of the handful of people who actually make the decision. ChrisA
In this case, the governance model for the Python language is being discussed.
This was the info I was missing, where is it discussed ? Not only on this list I assume ^^
Upvotes and downvotes don't mean anything. [...]
Yes, that's why random people wouldn't vote. But like, voting between like the 10 core devs where they all have the same importance, that does help for choosing "to merge or not to merge", isn't it ?
On Fri, Nov 2, 2018 at 11:26 AM Robert Vanden Eynde <robertve92@gmail.com> wrote:
In this case, the governance model for the Python language is being discussed.
This was the info I was missing, where is it discussed ? Not only on this list I assume ^^
There are a number of PEPs in the 8000s that would be worth reading.
That is just one of the possible options - that decisions are made by vote. ChrisA
On Fri, Nov 02, 2018 at 10:48:22AM +0200, Serhiy Storchaka wrote:
I disagree that "the mailing list was not really into it". So far, I count 12 people who responded to the original post by Giampaolo. By my count, I see: * five people in favour; * three people against, or see no need for it; * four people I can't tell if they are for or against, (possibly neutral?) [1] I know that adding features isn't decided by majority vote, but it seems clear to me that there is a substantial set of Python users, perhaps a majority, who would find this feature useful and more obvious than the alternatives. [Serhiy]
It is better to not do this. PEP 572 was initially written with the intention to be rejected.
Sounds like an excellent reason to write a PEP :-) There are some issues that ought to be addressed: - The status quo is easy to get wrong: # I've written this. More than once. L.pop(idx) if idx < len(L) else default is wrong if there is any chance of idx being negative. - The more common case of popping from the front of the list is not thread-safe: L.pop() if L else default - This clever trick is probably thread-safe (I think...) but it is wasteful and inefficient: (L or [default]).pop() and it isn't obvious how to adapt it efficiently if you need to pop from an arbitrary index. I came up with this: (L[idx:idx+1] or [default]).pop() but it is doubly wrong. - The obvious thread-safe EAFP idiom is a try...except statement, so it needs to be wrapped in a helper function to use it in expressions. That adds more overhead. The proposed .get(idx, default=x) and .pop(idx, default=x) signatures ought to be obvious and unsurprising to any moderately experienced Python programmer. These aren't complicated APIs. On the other hand: - I'm not volunteering to do the work (I don't know enough C to write a patch). Unless somebody has a patch, we can't expect the core devs who aren't interested in this feature to write it. (Hence, status quo wins a stalemate.) [1] "What makes a man turn neutral? Lust for gold? Power? Or were you just born with a heart full of neutrality?" -- Captain Zapp Brannigan -- Steve
On 11/2/18 7:39 AM, Anders Hovmöller wrote:> little improvement. I'm having a hard time seeing a real use case. Giampaolo's original post contains this link: https://github.com/giampaolo/psutil/blob/d8b05151e65f9348aff9b58da977abd8cac... Yuck (from an aesthetics standpoint, not a functional standpoint). :-) There's an impedance mismatch between the data, which is structured and has changed apparently arbitrarily between Linux releases, and the return value of string.split, which is an ordered collection. This code effectively hides that mismatch and yields Python tuples, which represent structured data. I can certainly see the desire for a simpler solution (for some definition of simpler), but how would adding a default parameter to list.pop make this code any simpler? Dan
On Fri, Nov 2, 2018 at 3:59 AM, Steven D'Aprano <steve@pearwood.info> wrote:
Well, it would be frustrating to have a feature accepted but not implemented, but the steps are separate. And it wouldn't have to be a core dev that implements it -- anyone with the C chops (not me :-) ) could do it. As an example, a good chunk of PEP 485 was implemented by someone else (I wrote the first prototype, but it was not complete), who I'm pretty sure is not a core dev. A core dev has to actually merge it, of course, and that is a bottleneck, but not a show stopper. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
I don't understand the rationale behind PEP 463's rejection. Guido says, "I disagree with the position that EAFP is better than LBYL, or "generally recommended" by Python. (Where do you get that?..."; but it's been in the official Python.org docs <https://docs.python.org/3/glossary.html> for a while and even provides a pretty good justification for why EAFP is preferable to LBYL (aside from the language calling EAFP "common", "clean", and "fast" that's notably absent from LBYL's description): "In a multi-threaded environment, the LBYL approach can risk introducing a race condition between “the looking” and “the leaping”. For example, the code, if key in mapping: returnmapping[key] can fail if another thread removes *key* from *mapping* after the test, but before the lookup. This issue can be solved with locks or by using the EAFP approach." Which brings me to the question: What happens when a PEP gets rejected? Is it final? Is there a process for reviving a PEP? I personally would love to have *both* more consistent methods on built-in classes AND exception handling expressions. I think the colon (and maybe 'except' keyword) could be replaced with an exclamation point: value = lst[2] except IndexError! "No value" or just: value = lst[2] IndexError! "No value" if that appeases the people who dislike the over-use of colons. A full exception list would have to be in parentheses which get's ugly, but would also be (I would wager) a less common form: dirlist.append(os.getcwd() (AttributeError, OSError as e)! os.curdir) That might need some work. I don't know if it's compatible w/ the compiler. It may have to start with "try" or something, but it seems pretty close to a workable solution. On Wed, Oct 31, 2018 at 4:42 AM Chris Angelico <rosuav@gmail.com> wrote:
On Wed, Nov 7, 2018 at 6:04 AM Abe Dillon <abedillon@gmail.com> wrote:
Which brings me to the question: What happens when a PEP gets rejected? Is it final? Is there a process for reviving a PEP?
It remains as a permanent document. No, that isn't final; and the process for reviving a PEP basically consists of answering the objections that led to its rejection. There have been a few cases where a proposal lies dormant for years before finally being accepted (such as the matrix multiplication operator). So if you want to do that, open a new thread, and specifically respond to the issues in the PEP - anything named as a reason for rejection, and anything else that you think ought to be improved. ChrisA
On Tue, Nov 06, 2018 at 01:03:54PM -0600, Abe Dillon wrote:
I can't comment on Guido's question about "generally recommended", but as for the first part, I agree: neither EAFP nor LBYL is "better", they are both appropriate under different circumstances. Sometimes one is clearer and more efficient than the other. The only time I would say that EAFP is clearly better is when LBYL introduces "Time Of Check To Time Of Use" bugs.
Which brings me to the question: What happens when a PEP gets rejected? Is it final? Is there a process for reviving a PEP?
Nothing is final-final. You can try opening a competing PEP, or take over as champion of the existing PEP (assuming Chris is willing to step aside). You ought to respond to the reasons given in the rejection. It's probably a good idea to gauge the chances of success by asking on Python-Ideas and Python-Dev first, to avoid the core devs saying "Oh give it up, it's not going to happen!" after you've wasted time trying to revise a rejected PEP. [...]
And I think that this is precisely the sort of syntax that prompted Guido to write many years ago that language design is not merely a problem-solving exercise. Aesthetics are important. This is not just a matter of finding an unused character or two and hammering it into the the language. That's how you get Perl, which is not a pretty language.
Seeing that syntax, the phrase that came to my mind was not so much "close to workable" and more "kill it with fire!". -- Steve
neither EAFP nor LBYL is "better", they are both appropriate under different circumstances. Sometimes one is
clearer and more efficient than the other.
One of the reasons LBYL is sometimes cleaner than EAPF is because it has more support from the language in the form of an expression which is what PEP 463 intends to change. The only time I would say that EAFP is clearly better is when LBYL
introduces "Time Of Check To Time Of Use" bugs.
It also puts the intent of the logic up front instead of requiring the reader to scroll through a preamble of edge-case checks to get to what the code is actually trying to do. I think that this is precisely the sort of syntax that prompted
Guido to write many years ago that language design is not merely a problem-solving exercise.
The sort of syntax that prompted that post was "precisely" multi-line lambdas. Guido explained that he tried to throw people off the idea of multi-line lambdas by posing it as an unsolvable puzzle (which people promptly solved) when really he just thought the concept of multi-line lambdas was flawed to begin with. I agree with him on that point. The whole point of a lambda is that, in certain cases, they allow you to write more expressive code by saying in-line exactly what you want to do. It only works if that action is easily expressed in a line: button.onClick(lambda: print("Hello!")) If it's a long an complicated bit of code, the expressiveness of lambda is lost and it makes more sense to give it a name and write: button.onClick(doThatComplicatedThing) I'm not trying to solve a puzzle that implements an anti-pattern (unless you have some argument for why expressionized try-except would be an anti-pattern). Aesthetics are important. This is not just a matter of finding an unused
character or two and hammering it into the the language.
Yeah. That's why I didn't just try to find an unused character and hammer it into the language without paying any regard to aesthetics. I find value = lst[2] except IndexError! "No value" to be pretty well in keeping w/ Python's aesthetics because raising an exception pretty naturally fits with an exclamation point, but of-course; aesthetics are subjective. I know what Perl is BTW, and share your distaste for it. Seeing that syntax, the phrase that came to my mind was not so much
"close to workable" and more "kill it with fire!".
Funny, that's exactly how I felt about the None-aware operators, only; I didn't reject the entire concept simply because I disliked the syntax choice. I simply rejected the syntax choice because I disliked the syntax choice... On Tue, Nov 6, 2018 at 5:21 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Thu, Nov 01, 2018 at 09:36:19PM +0200, Serhiy Storchaka wrote:
That's not an expression, so there are limits to where and when you can use it. What we need is a helper function that wraps that, called "pop". And since this seems to be reoccuring request going back nearly 20 years now: https://mail.python.org/pipermail/python-dev/1999-July/000550.html https://stackoverflow.com/questions/31216428/python-pop-from-empty-list as is the more general get(list, index, default=None) helper: https://stackoverflow.com/questions/2574636/getting-a-default-value-on-index... https://stackoverflow.com/questions/2492087/how-to-get-the-nth-element-of-a-... https://stackoverflow.com/questions/5125619/why-doesnt-list-have-safe-get-me... https://stackoverflow.com/questions/17721748/default-value-for-out-of-bounds... we could save people from having to re-invent the wheel over and over again by add them to a new module called "things_that_should_be_list_methods_but_arent.py" *wink* -- Steve
31.10.18 13:08, Antoine Pitrou пише:
list.pop() corresponds two dict methods. With argument it corresponds dict.pop(). But there are differences: dict.pop() called repeatedly with the same key will raise an error (or return the default), while list.pop() will likely return other item. Without argument it corresponds dict.popitem() which doesn't have an optional default.
Does it make sense to draw some sort of parallel between next(myiterator, default="whatever") and mylist.pop(default="whatever")? They exhaust the iterator/list then start emitting the default argument (if provided). On Thu, Nov 1, 2018 at 2:46 PM Serhiy Storchaka <storchaka@gmail.com> wrote:
Yep that's what I just did in my previous mail. """ I think the same way about set.pop, list.pop. About .index I agree adding default= would make sense but that's not exactly the same thing as the others. """ Being picky: "TypeError: next() takes no keyword arguments", that's next(myierator, "whatever") ;)
On Fri, Nov 2, 2018 at 4:45 AM Serhiy Storchaka <storchaka@gmail.com> wrote:
I think there is one more important difference between dict and list. dict has .get(key[, default]), but list doesn't have it. If we add only `list.pop([default])`, it is tempting that using it even when they don't have to remove the item. Unnecessary destructive change is bad. It reduces code readability, and it may create hard bug. If this proposal is adding `list.get([index[, default]])` too, I still -0. I don't know how often it is useful. Regards, -- INADA Naoki <songofacandy@gmail.com>
participants (18)
-
Abe Dillon -
Anders Hovmöller -
Antoine Pitrou -
Chris Angelico -
Chris Barker -
Dan Sommers -
Eric Fahlgren -
Ethan Furman -
Giampaolo Rodola' -
INADA Naoki -
Michael Selik -
Michel Desmoulin -
Nick Timkovich -
Nicolas Rolin -
Robert Vanden Eynde -
Ron Reiter -
Serhiy Storchaka -
Steven D'Aprano