Fwd: Null coalescing operator
Sorry, I sent this accidentally as private reply, then tried to fix it on phone. The latter produced horrible formatting. Please just read this version. On Sat, Sep 10, 2016 at 4:10 PM, Guido van Rossum <guido@python.org> wrote:
So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to `x?.bar`... Color me unconvinced.
No, I'm offering a more realistic use pattern: for x in get_stuff(): x = NoneCoalesce(x) # ... bunch of stuff with x ... # ... more stuff with nested keys or attributes ... x2 = x.foo x3 = x.bar.baz[x2] x4 = x(x.val) result = x3(x4) As a less ugly alternative in the fairly uncommon case that you want None coalescing as the behavior of getting attributes, keys, call values, etc. that may or may not be available (AND where you don't want to wrap all of those access patterns in one try/except block). In contrast, the ugly version of even this pretty simple toy code with the hypothetical syntax would be: for x in get_stuff(): # ... bunch of stuff with x ... # ... more stuff with nested keys or attributes ... x2 = x?.foo x3 = x?.bar?.baz?[x2] x4 = x?(x?.val) result = x3?(x4) This second case looks absolutely awful to me. And real world uses, if implemented, would quickly get much worse than that. Yours, David... -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
There seems to be a major misunderstanding here. A None-coalescing operator is not for catching AttributeError, it's a shortcut similar to "a or b" except that it checks for "a is None" rather than bool(a). On Sat, Sep 10, 2016 at 4:38 PM, David Mertz <mertz@gnosis.cx> wrote:
-- --Guido van Rossum (python.org/~guido)
On Sep 10, 2016 4:45 PM, "Guido van Rossum" <guido@python.org> wrote:
That's exactly what the wrapper does. Except it converts all the regular operators into their None-coalescing versions by putting the extra checks into the wrapped object itself. Now admittedly, this DOES mean that the behavior of operations is somewhat different depending on whether they are wrapped objects or not. False-like is a different thing than None-like. This really MUST BE essentially a way a catching AttributeErrors though. With the proposed syntax 'x?.foo?.bar' will resolve even if x has no 'foo'. So 'x?.'foo' has to be something special. I guess that special thing could be a 3.7-style None that responds to new operators, but that's essentially still *wrapping* a 3.6-style None. In my mind, as I say, the question marks look ugly, especially when repeated in chained operations (attribute, call, item get). But even if they didn't feel bad visually, I don't believe the use case is common enough to warrant dedicated syntax. Even if my keyboard had some character I thought was beautiful and intuitive for that meaning, it's still an extra cognitive burden to distinguish the plain from None-coalescing versions of every operation, especially for learners. Another problem is that the question mark still doesn't actually get the special 'a or b' behavior. For that you still need 'a if a is not None else b'. Or I guess, in concept, 'a ?or b'. For what it's worth, the wrapper gives you the special 'a or b' semantics by casting non-Nones as truthy... But again, 'a' has to have been wrapped first.
On Sat, Sep 10, 2016 at 4:38 PM, David Mertz <mertz@gnosis.cx> wrote:
Sorry, I sent this accidentally as private reply, then tried to fix it
on the
On Sat, Sep 10, 2016 at 5:23 PM, Guido van Rossum <guido@python.org> wrote:
No. PEP 505 actually solves the problem without ever catching AttributeError. Please read it.
I read it again (I did a year ago, but reviewed it now). I hadn't been thinking that the *mechanism* of a new None-coalescing operator would actually be catching an exception. It could (and should) work differently if it becomes syntax. What I was getting at with "essentially" was that it would *do the same thing* that an AttributeError does. That is, if `x.foo` can't be evaluated (i.e. x doesn't have an attribute 'foo'), then access is informally "an error." The hypothetical "x?.foo" catches that "error" and substitutes a different value. The particular implementation under-the-hood is less important for most programmers who might use the construct (and I think documentation would actually give an informal equivalent as something similar to what I put in the NoneCoalesce class). -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 2016-09-11 02:02, David Mertz wrote:
x?.foo would lookup attribute 'foo' _unless_ x was None, in which case it would return None. It's simply: None if x is None else x.foo This means that None?.__str__() would return None, not 'None'. (None has an attribute called '__str__', and None.__str__() returns 'None', but it would not be looked up because None is, well, None.)
Ok, I have been thinking of the behavior too broadly. I realize now that `x?.foo` might still simply raise an AttributeError if x is neither None nor a thing with a foo attribute. The class I wrote is definitely too aggressive for the behavior described. On the other hand, by being narrower in behavior there feels like even less motivation for new syntax. On Sep 10, 2016 6:29 PM, "MRAB" <python@mrabarnett.plus.com> wrote:
To the contrary. I read and write code that performs explicit checks for None all the time. Catching AttributeError is often a code smell or at least a measure of last resort. On Sat, Sep 10, 2016 at 6:46 PM, David Mertz <mertz@gnosis.cx> wrote:
-- --Guido van Rossum (python.org/~guido)
How much of the time is a branch of the None check a single fallback value or attribute access versus how often a suite of statements within the not-None branch? I definitely check for None very often also. I'm curious what the breakdown is in code I work with. On Sep 10, 2016 7:10 PM, "Guido van Rossum" <guido@python.org> wrote:
For kicks I looked at the code in a commercial product that we are open sourcing very soon at Continuum Analytics. It has about 27k lines of Python and Cython. I checked it just by running `grep -C3 'if.*None'` over the source code and eyeballing. Obviously, this code might be refactored if other syntax was available. And I personally wrote only a small part of the code, so I might write it differently either way. But looking through, the uses of 'is (not) None' I found fell into a few categories. Some were suites of statements under the 'foo is None' branch. Those would have to remain suites. Some are single line returns, other single line raise. Those I call 'value' are the ones that would lend themselves to the 'bar = foo ?? baz' style. The ones I call 'attribute' are the cases where we test for non-None before accessing an attribute of the object. The thing that I found surprisingly common was cases where a compound condition was used, e.g. 'if x is None and y > 37'. Those might be possible to refactor, but cannot be directly rewritten in the coalescing style. - Suite: 75 - Return: 21 - Raise: 13 - Value: 46 - Attribute: 2 - Compound condition: 25 On Sat, Sep 10, 2016 at 7:15 PM, David Mertz <mertz@gnosis.cx> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sat, Sep 10, 2016 at 6:02 PM, David Mertz <mertz@gnosis.cx> wrote:
That's not a good way to think about it. This new operator is only checking for None and not actually checking for AttributeErrors. Consider: (3).x # AttributeError {}.x # AttributeError None.x # AttributeError (3)?.x # still AttributeError {}?.x # still AttributeError None?.x # None And also: None.__class__ # <type 'NoneType'> None?.__class__ # None And it's certainly not the case that those values don't accept any attributes: (3).real # 3 {}.values # <built-in method ...> None.__class__ #<type 'NoneType'> --- Bruce Check out my puzzle book and get it free here: http://J.mp/ingToConclusionsFree (available on iOS)
Both this discussion, PEP 505, and the one a year ago, tend to mix up 2 related but separate proposals: w (A) Add a None-coalescing operator (like C# a ?? b, what you would write in Python as "a or b" if it didn't have the falsy gotcha) (B) Add some None-aware navigation operators ( The "?.", "?()", "?[]", or what you would write in python as "a and a.attribute" if it didn't have the falsy gotcha) Both are things that can be already done in python, so the purpose here is to add some convenience (aka "syntax sugar"). IMO, this kind of syntax sugar proposals should be weighed with the frequency of the coding pattern where the sugar can be applied. And from the stats presented in PEP-505 (B) is one order of magnitude less usual than (A); that matches most of the examples I see in the threads and FWIW my personal experience. So, as a counterproposal I would like to suggest: * Add an "a ?? b" operator which is equivalent to "a if a is None else b" (but evaluating a once) * Do not add none-aware navigation; in the less usual scenario where you need to do it AND ALSO the "and" operator is not usable (it frequently is, given that by default classes are truish), well, you can use a ternary operator * I don't care if it's an alternate syntax (I'm surprised nobody suggested "||" which is also used in other languages for similar purpose) Would this satisfy most of the people requesting this? (and, would it satisfy the people making the decision?) On Sun, Sep 11, 2016 at 4:45 AM, Bruce Leban <bruce@leban.us> wrote:
-- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Sun, Sep 11, 2016 at 12:44 PM, Daniel Moisset <dmoisset@machinalis.com> wrote:
Both this discussion, PEP 505, and the one a year ago, tend to mix up 2 related but separate proposals:
I don't think there's that much of a mix-up. PEP 505 clearly describes each proposal separately and even gives a choice to accept or reject each one separately.
(A) Add a None-coalescing operator (like C# a ?? b, what you would write in Python as "a or b" if it didn't have the falsy gotcha)
https://www.python.org/dev/peps/pep-0505/#none-coalescing-operator
https://www.python.org/dev/peps/pep-0505/#none-aware-attribute-access-operat...
I can't argue here yet. Honestly I looked at some examples; for those where it wasn't instantly clear that a None-coalescing operator would *not* help, I found it hard to figure out how to rewrite it. E.g. this one -- quick, is there a better way? return "Illegal Argument" + (self.message is not None and (": " + self.message) or "") I think the answer is, if we had both None-coalescing (??) and None-severing (!!) it could be written as follows: return "Illegal Argument" + ((self.message !! (": " + self.message)) ?? "") but it took me way too long to prove that to myself.
Honestly the one thing that makes `?.` attractive is that it's easier than the None-coalescing and -severing operators to grasp at an intuitive level. If "foo.bar" raises "AttributeError: 'NoneType' object has no attribute 'foo'" then try again with "foo?.bar". It's surprising how often that will work!
* I don't care if it's an alternate syntax (I'm surprised nobody suggested "||" which is also used in other languages for similar purpose)
Interestingly, after analyzing the above example I desperately want to write it as return "Illegal Argument" + (self.message && (": " + self.message) || "") Note that I already know the relative priorities of && and ||, so I can drop a set of parentheses.
Would this satisfy most of the people requesting this? (and, would it satisfy the people making the decision?)
Personally, after the above example, I'm less excited about ??/!! or ||/&&, and more excited about `?.` -- so it doesn't satisfy me. -- --Guido van Rossum (python.org/~guido)
On Sun, Sep 11, 2016 at 12:44 PM, Daniel Moisset <dmoisset@machinalis.com> wrote:
I readily confess that my initial comments here had a think-o about just what was being discussed, notwithstanding having also followed the discussion a year ago. I somehow had in mind that "None-coalescing" meant something like "get to None on failure" (i.e. *coalesce* to) I think the reason I thought wrong was because the second set of syntax really encourages that wrong way of thinking. I don't find `a ?? b` particularly ugly, even if I don't entirely want it. Similarly if it were spelled `a || b`. Both of those feel easy to conceptualize and teach as "another kind of short-circuiting, similar to boolean operators." Maybe an actual word is better? `a ifNone b`? `a failto b`? Falsey is similar to None-like (I know it's actually None-identical), so the shortcut idea is familiar. So I'm only -0 on that idea. However, the None-aware navigation operators suggest something very different. `a?.foo` MIGHT BE an attribute access, or it might not be. Likewise, `a?[x]` might be an item get or it might not be. And `a?(y)` might or might not be a function call. Obviously, if some or all of those forms are not added they are simply syntax errors. What operation happens when we use these syntax forms is "it depends"... that answer feels less straightforward than "We call `a.__getattr__('foo')`" (or `a.__getitem__(x)` or `a.__call__(y)`, as the case may be). Even if there were some characters I found attractive for these "it depends" operations, that would introduce a needless non-uniformity into the semantics of Python. In contrast, a fully spelled out ternary doesn't give me this uneasiness. Of course the following expression might be one thing or the other, but that's more obvious in the ternary syntax (also more general, you can put *any* values on both branches of the ternary): None if a is None else a.foo -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sun, Sep 11, 2016 at 6:00 PM, David Mertz <mertz@gnosis.cx> wrote:
None if a is None else a.foo
This is the crux of the matter to me. It's just too verbose, and the `if` and `else` keywords are lost in the noise of all the other words on the line. Plus the big win when it applies) is that if `a` is in fact something more complex, like `f(a)`, repeating it twice sounds like a performance penalty, and that's where `f(a)?.foo` really shines. -- --Guido van Rossum (python.org/~guido)
On Sun, Sep 11, 2016 at 6:11 PM, Guido van Rossum <guido@python.org> wrote:
The non-repetition is certain a plus, I readily confess. It's not only performance even; `f()` might not be a pure function. Silly example:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
I messed up my answer and replied to one person instead of the list, so I'll post it again. There is also an alternative to this operator, and it's allowing a shortcut to do: try: val = do_thing() except ThingError: val = "default" In the form of: val = do_thing() except ThingError: "default" I was debated, and rejected, but I feel like mentioning it again because it has some strong benefits. First, it handles the null coalescing very quite well: val = obj.foo.bar.hey except AttributeError: "default" But it also can deal with many common operations in Python without the need to add more operators or variants: val = my_list[0] except IndexError: "default" val = iterable[0] except TypeError: next(iter(iterable)) val = int(param) except ValueError: man.nan It's quite readable, in the same vein of val = foo if bar else "default", but also familiar since it's using known keyword. And it doesn't require to add a new operator support in the parser. Another serious benefits is that it fits current use cases, AND futur use cases. Indeed, since it leverages Python exception mechanism, any lib implementing a clean error model can immediately let the users benefit from it without having to implement, tests and document lots of helper methods with "default" keywords and the like, while not being limited to a set of values some operators would only care about, such as None. Plus, since EAFP is a popular and handy pattern, it makes sense. At last, it has the same characteristic as the null coalescing operator: it's lazy, and hence has a small performance interest too compared to functional equivalent. Did I mention it's also easy to expend to a full try/except when you need something more complicated ? And then you benefit from else and finally immediately. I already have many code that would benefit from such a syntax, and I'd like to hear again what you think about it. Le 11/09/2016 à 21:44, Daniel Moisset a écrit :
On 09/12/2016 12:05 AM, Michel Desmoulin wrote:
+1 There are many places in my code where this would clean things up a bit. Not having it is like not having list comps or not having the ternary if-else -- possible, but it would be much nicer to have it. -- ~Ethan~
On 12 September 2016 at 09:05, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
I like this idea, I would propose a (maybe crazy) addition to it. What about a special exception NoneError, that will catch TypeError, AttributeError etc. but only when it was caused by None(), None.attr, None[1], etc. With this one can write: x = a.b()[0] except NoneError: 'default' without a risk of catching other (unrelated) exceptions. -- Ivan
On 12/09/2016 11:01, Ivan Levkivskyi wrote:
Assuming you can't break existing code that already traps TypeError, AttributeError, etc., I don't see how you can do this without having separated kinds of NoneError which were subclasses of TypeError, AttributeError, etc. Rob Cliffe
On Tue, Sep 13, 2016 at 12:03 AM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
class NoneError(Exception): pass class TypeNoneError(TypeError, NoneError): pass class AttributeNoneError(AttributeError, NoneError): pass Now you can catch NoneError to catch None.xyz, or AttributeError to catch foo.xyz for any foo. I don't think it's good, but it's possible. ChrisA
For the record, I still really don't like PEP 463. We should strive to catch fewer exceptions, not make it easier to catch them. On Mon, Sep 12, 2016 at 7:09 AM, Chris Angelico <rosuav@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On 12/09/2016 16:37, Guido van Rossum wrote:
Also I don't see the connection between the first sentence and the second. Exception-catching expressions as per PEP 463 just give you a more concise way of of doing something you can do with try+except. Do you mean it would make it easier to catch exceptions because you have to type fewer characters, and that this would be a bad thing? Thanks, Rob Cliffe
On Mon, Sep 12, 2016 at 5:45 PM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
(If that means "Leap Before You Look", yes. :-)
Yeah, that's exactly my point. PEP 463 gives you a shorter way to catch an exception, so it gives you less motivation to find a way to write your code (or define your API) that doesn't involve catching exceptions. But APIs involving exceptions are often inferior to APIs that don't require exception catching. (Yes, I am aware of __next__() raising StopIteration -- but that API design usually doesn't require you to catch it.) -- --Guido van Rossum (python.org/~guido)
On 13/09/2016 04:43, Guido van Rossum wrote:
You surprise me. I thought LBYL and EAFP were both approved Python idioms, in some cases one being better, in some cases another, choice to be made on the merits of each case (or the author's preference). I certainly use both (and sometimes time both to see which is faster). Now it sounds as if you're trying to impose a style guide on the world by discouraging the EAFP. And wasn't the discussion general, not about APIs specifically? Best wishes, Rob Cliffe
On 13 September 2016 at 21:15, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
Which is preferable depends greatly on context of use, which is why you'll find a lot of Python APIs offer both forms - it's not *just* a matter of inheriting the exceptionless version from C, and then later adding a Python version that gives an exception instead of None or a user-supplied default value. It's similar to why IEEE754 defines both quiet NaN *and* signalling NaN - which one you want depends on what you're doing. In web servers, for example, you'll often have lots of fields where "not present" is a perfectly acceptable return value. For those, APIs that just return None for unknown entries are very handy, which is why SQL Alchemy offers both ".first()" and "one()", which mainly differ in how and when they throw an exception, rather than what they do when they succeed. However, blindly catching *all* exceptions from a complex subexpression is rarely the right thing to do, so APIs that only offer "this may throw exceptions during normal operation under these circumstances" without a convenience wrapper that does the exception handling for you can end up being a pain to work with. PEP 463 makes those APIs less painful to deal with, but at the cost of encouraging overly broad exception handlers. By contrast, fixing APIs on a case-by-case basis puts the design burden where it can do the most good: on the API designer, who can make the scope of the exception handling suitably narrow *inside* the API implementation, rather than being limited to wrapping the entire API call in try/except. Cheers, Nick. P.S. There are also some use cases where Look-Before-You-Leap is inherently subject to race conditions, and for those, exceptions are the only reliable signaling mechanism. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I doubt very much it will be used for very complexe cases. Just like comprehensions or ternary expressions, they are a good fit for some specific use cases, and people will quickly catch on which one. You rarely see nested comprehensions or ternary expressions while it's possible to do so, because the Python communality values tend to limit abuses. It will be the same for this. You won't see very complex usages, mostly things like: val = foo[-1] except IndexError: "bar" doh = val.attr.other except AttributeError: "default" Those would already be coded with something similar (or maybe a chain of getattr()) or next(iter()). It's not a huge risk. But it's a huge convenience. Le 13/09/2016 à 13:37, Nick Coghlan a écrit :
On Tue, Sep 13, 2016, at 08:44, Michel Desmoulin wrote:
It occurs to me that if lambda were more lightweight [whether it's the syntax or the implementation that's the problem is unclear], not only this but also the ternary operator could have been functions. val = iexcept(lambda: foo[-1], IndexError, lambda: "bar")
On Tue, Sep 13, 2016, at 07:37, Nick Coghlan wrote:
P.S. There are also some use cases where Look-Before-You-Leap is inherently subject to race conditions,
Which use cases *aren't*?
and for those, exceptions are the only reliable signaling mechanism.
It's entirely possible to design a non-exception interface without race conditions. dict.get is *almost* a suitable example of such a method. Remember, at the C level in CPython, throwing an exception *is* returning a value [typically (PyObject*)NULL for interfaces that otherwise return an object], along with setting the error indicator. The only missing piece is an "error value" outside the set of valid values. Guido's argument here seems to be that exception-based EAFP is not pythonic.
On Wed, Sep 14, 2016 at 12:14 AM, Random832 <random832@fastmail.com> wrote:
Ones in which no external force can affect things. For example: def frob(spam): if spam is None: print("Frobbing nothing") else: print("Frobbing some spam: ", spam) ... You can safely assume that locals won't be changed between the 'is None' check and the print. I suppose someone could mess around with thread call stacks, but that'd be seriously insane. ChrisA
On 13/09/2016 12:37, Nick Coghlan wrote:
"Guido's argument here seems to be that exception-based EAFP is not pythonic." [snip] try+except block. Of course result = (myList[0] except Exception: MyDefault) is poor code (presumably it should have been written "except IndexError"), but so is try: result = myList[0] except Exception: result = MyDefault ISTM you're giving an exception-catching dog a bad name and hanging him. Or have I missed something? And sorry to repeat myself, but we seemed to be having a *general* discussion about null-coalescing operators, which moved on to PEP 463, then suddenly Guido is talking about APIs. A lot of the time when I'm coding, I'm not writing an API, just a program to get a job done. No doubt, exceptions should be discouraged *in APIs*, but that doesn't make exceptions, or EAFP, a bad idea per se. I really don't mean this post to sound hostile, and I'm sorry if it comes across a bit that way. I'm just surprised at what I'm hearing, and to be honest, I have a soft spot for PEP 463. Best wishes Rob Cliffe
On Wed, Sep 14, 2016 at 1:27 AM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
PEP 463, by its nature, is talking about the language. Language features govern and advise API design, which then changes how you write "a program to get a job done", so it does actually have strong bearing - just via a couple of levels of indirection. ChrisA
On 2016-09-13 16:27, Rob Cliffe wrote:
[snip] I think the point is that exceptions are for, well, exceptional processing, not normal processing. Code should be written on the assumption that everything works normally, e.g. division always returns a result. _Occasionally_ division will fail, such as attempting to divide by zero, but that should be uncommon. If something pretty much always succeeds, use an exception for failure, but if it sometimes succeeds and sometimes fails, check explicitly for success or failure.
This general API discussion is veering off-topic for the subject line. If people would like to continue to discuss it then please start a new thread (if you feel it impacts your view of a null-coalescing operator then please discuss in the new thread and come back to this one). On Tue, 13 Sep 2016 at 11:25 MRAB <python@mrabarnett.plus.com> wrote:
On 09/12/2016 08:37 AM, Guido van Rossum wrote:
For the record, I still really don't like PEP 463. We should strive to catch fewer exceptions, not make it easier to catch them.
I certainly agree with the first part, slightly reworded: we should strive to generate fewer exceptions that we have to catch. I disagree with the second part: being able to condense four lines of code (1 for try, 1 for except, 1 for the attempt, and 1 for recovery) in to one line of code seems like a win. I know I find it frustrating when my choice is between the 4-line boiler-plate try/except, or an equally verbose and ugly multi-line non-exception generating stanza. Anyway, my two cents worth. If you collect them all for this subject I think I owe you a dime. ;) -- ~Ethan~
On 12/09/2016 08:05, Michel Desmoulin wrote:
+1, you're preaching to the converted. This is PEP 463, "Exception-catching expressions". Except that the PEP required parentheses around a whole exception-catching expression, for reasons that are not clear to me, i.e. val = (my_list[0] except IndexError: "default") rather than val = my_list[0] except IndexError: "default" Rob Cliffe
On Mon, Sep 12, 2016 at 1:05 AM, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
Note that there's a subtle difference here when multiple lookups are involved. Given: def f(spam): return spam().eggs().ham With null-coalescing: def f(spam): return spam()?.eggs()?.ham This is roughly equivalent to: def f(spam): _spam = spam() try: eggs = _spam.eggs except AttributeError: return None _eggs = eggs() try: return _eggs.ham except AttributeError: return None With PEP 463 it doesn't work out so well. The "obvious" spelling would be: def f(spam): return (spam().eggs().ham except AttributeError: None) This is roughly equivalent to: def f(spam): try: return spam().eggs().ham except AttributeError: return None Note how it's different. For one thing, it could mask AttributeError coming from the calls. For another, you no longer explicitly identify which lookups to handle. I would expect both to lead to subtle bugs, whereas with null-coalescing you don't have those problems. -eric
On 12 September 2016 at 21:47, Eric Snow <ericsnowcurrently@gmail.com> wrote:
From previous explanations in this thread, that's *not* the behaviour at all.
If I understand the proposal, f is actually intended to be equivalent to: def f(spam): spam_val = spam() if spam_val is None: return None eggs_val = spam_val.eggs() if eggs_val is None: return None return eggs_val.ham Personally, I find it pretty worrying that there's this persistent confusion over what the proposed syntax means. Sure, it's explained in the PEP and proposal, but if it gets implemented it'll be in the docs. And yet people don't seem to read any of those - and their intuition of what the construct does is wrong. IMO, the syntax may well be useful, but if we don't address the problem that what people *think* it means isn't what it actually means, then it'll do more harm than good. Paul
On 09/12/2016 02:13 PM, Paul Moore wrote:
On 12 September 2016 at 21:47, Eric Snow wrote:
Perhaps the bumper-sticker explanation is: None-coalescing is not to /recover/ from an AttributeError, but to _prevent it_ in the first place -- but only when the base object is already None. Okay, several bumper stickers. ;) -- ~Ethan~
On Mon, Sep 12, 2016 at 3:13 PM, Paul Moore <p.f.moore@gmail.com> wrote:
From previous explanations in this thread, that's *not* the behaviour at all.
Gah, you're right. I knew better too. The diversion back to PEP 463 crossed some wires in my brain. :)
...which is even less related to PEP 463.
The tricky bit is the syntactic association with attribute lookup. However, I don't think it's quite so confusing. As noted above, my confusion was mostly due to the recent mention of PEP 463. -eric
On 13 September 2016 at 07:13, Paul Moore <p.f.moore@gmail.com> wrote:
Right, there are two quite reasonable interpretations for what a conditional attribute lookup does, and the "catch and ignore AttributeError" case is going to be more familiar to most Pythonistas, as that's the way getattr() works when given a default value. Consider the case of chained attribute lookup as a function using a "None" default to getattr(): def attr_chain_abort_on_missing(base, *attrs): result = base for attr in attrs: if result is None: break result = getattr(result, attr, None) return result vs the actually proposed semantics: def attr_chain_abort_only on_none(base, *attrs): result = base for attr in attrs: if result is None: break result = getattr(result, attr) return result In the latter version, AttibuteError escapes - it's only when the original value is None, or an attribute exists and is None that the iteration bails out early without raising an exception.
Agreed, and I think a key indicator for that would be whether or not people that saw: result = obj?.first?.second?.third agreed on whether or not it could raise AttributeError or TypeError. Any prospective PhD students in the audience looking for a language usability question to study before 3.7b1 rolls around in 2018? :) Cheers, Nick. P.S. I'll note that the *upside* I see to (this part of) the proposal is that it would implement a *very* fine-grained "is not None" check, where the alternative in real code would often be an overly broad exception handling clause, like: try: result = obj.first().second.third except (TypeError, AttributeError): return None rather then actually spelling out the fine-grained checks properly. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Sep 10, 2016, at 20:15, David Mertz wrote:
To put it more explicitly, more similar to "a and a.b"
Why? I think you're confusing this proposal for something else. This is for treating x is None specially, not simply catching AttributeError to deal with the fact that None.foo doesn't exist.
On Sat, Sep 10, 2016, at 19:38, David Mertz wrote:
x2 = x?.foo
x3 = x?.bar?.baz?[x2]
A. if you're doing three different things with x, why are you using this instead of wrapping it in an if statement? B. Under some of the proposals, unless x.bar might be None independently of x not being None, this would just be "x?.bar.baz[x2]
x4 = x?(x?.val)
C. Under some of the proposals, the inner contents of the call brackets aren't evaluated if x is None, so you don't need to use x?.val here.
result = x3?(x4)
There seems to be a major misunderstanding here. A None-coalescing operator is not for catching AttributeError, it's a shortcut similar to "a or b" except that it checks for "a is None" rather than bool(a). On Sat, Sep 10, 2016 at 4:38 PM, David Mertz <mertz@gnosis.cx> wrote:
-- --Guido van Rossum (python.org/~guido)
On Sep 10, 2016 4:45 PM, "Guido van Rossum" <guido@python.org> wrote:
That's exactly what the wrapper does. Except it converts all the regular operators into their None-coalescing versions by putting the extra checks into the wrapped object itself. Now admittedly, this DOES mean that the behavior of operations is somewhat different depending on whether they are wrapped objects or not. False-like is a different thing than None-like. This really MUST BE essentially a way a catching AttributeErrors though. With the proposed syntax 'x?.foo?.bar' will resolve even if x has no 'foo'. So 'x?.'foo' has to be something special. I guess that special thing could be a 3.7-style None that responds to new operators, but that's essentially still *wrapping* a 3.6-style None. In my mind, as I say, the question marks look ugly, especially when repeated in chained operations (attribute, call, item get). But even if they didn't feel bad visually, I don't believe the use case is common enough to warrant dedicated syntax. Even if my keyboard had some character I thought was beautiful and intuitive for that meaning, it's still an extra cognitive burden to distinguish the plain from None-coalescing versions of every operation, especially for learners. Another problem is that the question mark still doesn't actually get the special 'a or b' behavior. For that you still need 'a if a is not None else b'. Or I guess, in concept, 'a ?or b'. For what it's worth, the wrapper gives you the special 'a or b' semantics by casting non-Nones as truthy... But again, 'a' has to have been wrapped first.
On Sat, Sep 10, 2016 at 4:38 PM, David Mertz <mertz@gnosis.cx> wrote:
Sorry, I sent this accidentally as private reply, then tried to fix it
on the
On Sat, Sep 10, 2016 at 5:23 PM, Guido van Rossum <guido@python.org> wrote:
No. PEP 505 actually solves the problem without ever catching AttributeError. Please read it.
I read it again (I did a year ago, but reviewed it now). I hadn't been thinking that the *mechanism* of a new None-coalescing operator would actually be catching an exception. It could (and should) work differently if it becomes syntax. What I was getting at with "essentially" was that it would *do the same thing* that an AttributeError does. That is, if `x.foo` can't be evaluated (i.e. x doesn't have an attribute 'foo'), then access is informally "an error." The hypothetical "x?.foo" catches that "error" and substitutes a different value. The particular implementation under-the-hood is less important for most programmers who might use the construct (and I think documentation would actually give an informal equivalent as something similar to what I put in the NoneCoalesce class). -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On 2016-09-11 02:02, David Mertz wrote:
x?.foo would lookup attribute 'foo' _unless_ x was None, in which case it would return None. It's simply: None if x is None else x.foo This means that None?.__str__() would return None, not 'None'. (None has an attribute called '__str__', and None.__str__() returns 'None', but it would not be looked up because None is, well, None.)
Ok, I have been thinking of the behavior too broadly. I realize now that `x?.foo` might still simply raise an AttributeError if x is neither None nor a thing with a foo attribute. The class I wrote is definitely too aggressive for the behavior described. On the other hand, by being narrower in behavior there feels like even less motivation for new syntax. On Sep 10, 2016 6:29 PM, "MRAB" <python@mrabarnett.plus.com> wrote:
To the contrary. I read and write code that performs explicit checks for None all the time. Catching AttributeError is often a code smell or at least a measure of last resort. On Sat, Sep 10, 2016 at 6:46 PM, David Mertz <mertz@gnosis.cx> wrote:
-- --Guido van Rossum (python.org/~guido)
How much of the time is a branch of the None check a single fallback value or attribute access versus how often a suite of statements within the not-None branch? I definitely check for None very often also. I'm curious what the breakdown is in code I work with. On Sep 10, 2016 7:10 PM, "Guido van Rossum" <guido@python.org> wrote:
For kicks I looked at the code in a commercial product that we are open sourcing very soon at Continuum Analytics. It has about 27k lines of Python and Cython. I checked it just by running `grep -C3 'if.*None'` over the source code and eyeballing. Obviously, this code might be refactored if other syntax was available. And I personally wrote only a small part of the code, so I might write it differently either way. But looking through, the uses of 'is (not) None' I found fell into a few categories. Some were suites of statements under the 'foo is None' branch. Those would have to remain suites. Some are single line returns, other single line raise. Those I call 'value' are the ones that would lend themselves to the 'bar = foo ?? baz' style. The ones I call 'attribute' are the cases where we test for non-None before accessing an attribute of the object. The thing that I found surprisingly common was cases where a compound condition was used, e.g. 'if x is None and y > 37'. Those might be possible to refactor, but cannot be directly rewritten in the coalescing style. - Suite: 75 - Return: 21 - Raise: 13 - Value: 46 - Attribute: 2 - Compound condition: 25 On Sat, Sep 10, 2016 at 7:15 PM, David Mertz <mertz@gnosis.cx> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sat, Sep 10, 2016 at 6:02 PM, David Mertz <mertz@gnosis.cx> wrote:
That's not a good way to think about it. This new operator is only checking for None and not actually checking for AttributeErrors. Consider: (3).x # AttributeError {}.x # AttributeError None.x # AttributeError (3)?.x # still AttributeError {}?.x # still AttributeError None?.x # None And also: None.__class__ # <type 'NoneType'> None?.__class__ # None And it's certainly not the case that those values don't accept any attributes: (3).real # 3 {}.values # <built-in method ...> None.__class__ #<type 'NoneType'> --- Bruce Check out my puzzle book and get it free here: http://J.mp/ingToConclusionsFree (available on iOS)
Both this discussion, PEP 505, and the one a year ago, tend to mix up 2 related but separate proposals: w (A) Add a None-coalescing operator (like C# a ?? b, what you would write in Python as "a or b" if it didn't have the falsy gotcha) (B) Add some None-aware navigation operators ( The "?.", "?()", "?[]", or what you would write in python as "a and a.attribute" if it didn't have the falsy gotcha) Both are things that can be already done in python, so the purpose here is to add some convenience (aka "syntax sugar"). IMO, this kind of syntax sugar proposals should be weighed with the frequency of the coding pattern where the sugar can be applied. And from the stats presented in PEP-505 (B) is one order of magnitude less usual than (A); that matches most of the examples I see in the threads and FWIW my personal experience. So, as a counterproposal I would like to suggest: * Add an "a ?? b" operator which is equivalent to "a if a is None else b" (but evaluating a once) * Do not add none-aware navigation; in the less usual scenario where you need to do it AND ALSO the "and" operator is not usable (it frequently is, given that by default classes are truish), well, you can use a ternary operator * I don't care if it's an alternate syntax (I'm surprised nobody suggested "||" which is also used in other languages for similar purpose) Would this satisfy most of the people requesting this? (and, would it satisfy the people making the decision?) On Sun, Sep 11, 2016 at 4:45 AM, Bruce Leban <bruce@leban.us> wrote:
-- Daniel F. Moisset - UK Country Manager www.machinalis.com Skype: @dmoisset
On Sun, Sep 11, 2016 at 12:44 PM, Daniel Moisset <dmoisset@machinalis.com> wrote:
Both this discussion, PEP 505, and the one a year ago, tend to mix up 2 related but separate proposals:
I don't think there's that much of a mix-up. PEP 505 clearly describes each proposal separately and even gives a choice to accept or reject each one separately.
(A) Add a None-coalescing operator (like C# a ?? b, what you would write in Python as "a or b" if it didn't have the falsy gotcha)
https://www.python.org/dev/peps/pep-0505/#none-coalescing-operator
https://www.python.org/dev/peps/pep-0505/#none-aware-attribute-access-operat...
I can't argue here yet. Honestly I looked at some examples; for those where it wasn't instantly clear that a None-coalescing operator would *not* help, I found it hard to figure out how to rewrite it. E.g. this one -- quick, is there a better way? return "Illegal Argument" + (self.message is not None and (": " + self.message) or "") I think the answer is, if we had both None-coalescing (??) and None-severing (!!) it could be written as follows: return "Illegal Argument" + ((self.message !! (": " + self.message)) ?? "") but it took me way too long to prove that to myself.
Honestly the one thing that makes `?.` attractive is that it's easier than the None-coalescing and -severing operators to grasp at an intuitive level. If "foo.bar" raises "AttributeError: 'NoneType' object has no attribute 'foo'" then try again with "foo?.bar". It's surprising how often that will work!
* I don't care if it's an alternate syntax (I'm surprised nobody suggested "||" which is also used in other languages for similar purpose)
Interestingly, after analyzing the above example I desperately want to write it as return "Illegal Argument" + (self.message && (": " + self.message) || "") Note that I already know the relative priorities of && and ||, so I can drop a set of parentheses.
Would this satisfy most of the people requesting this? (and, would it satisfy the people making the decision?)
Personally, after the above example, I'm less excited about ??/!! or ||/&&, and more excited about `?.` -- so it doesn't satisfy me. -- --Guido van Rossum (python.org/~guido)
On Sun, Sep 11, 2016 at 12:44 PM, Daniel Moisset <dmoisset@machinalis.com> wrote:
I readily confess that my initial comments here had a think-o about just what was being discussed, notwithstanding having also followed the discussion a year ago. I somehow had in mind that "None-coalescing" meant something like "get to None on failure" (i.e. *coalesce* to) I think the reason I thought wrong was because the second set of syntax really encourages that wrong way of thinking. I don't find `a ?? b` particularly ugly, even if I don't entirely want it. Similarly if it were spelled `a || b`. Both of those feel easy to conceptualize and teach as "another kind of short-circuiting, similar to boolean operators." Maybe an actual word is better? `a ifNone b`? `a failto b`? Falsey is similar to None-like (I know it's actually None-identical), so the shortcut idea is familiar. So I'm only -0 on that idea. However, the None-aware navigation operators suggest something very different. `a?.foo` MIGHT BE an attribute access, or it might not be. Likewise, `a?[x]` might be an item get or it might not be. And `a?(y)` might or might not be a function call. Obviously, if some or all of those forms are not added they are simply syntax errors. What operation happens when we use these syntax forms is "it depends"... that answer feels less straightforward than "We call `a.__getattr__('foo')`" (or `a.__getitem__(x)` or `a.__call__(y)`, as the case may be). Even if there were some characters I found attractive for these "it depends" operations, that would introduce a needless non-uniformity into the semantics of Python. In contrast, a fully spelled out ternary doesn't give me this uneasiness. Of course the following expression might be one thing or the other, but that's more obvious in the ternary syntax (also more general, you can put *any* values on both branches of the ternary): None if a is None else a.foo -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sun, Sep 11, 2016 at 6:00 PM, David Mertz <mertz@gnosis.cx> wrote:
None if a is None else a.foo
This is the crux of the matter to me. It's just too verbose, and the `if` and `else` keywords are lost in the noise of all the other words on the line. Plus the big win when it applies) is that if `a` is in fact something more complex, like `f(a)`, repeating it twice sounds like a performance penalty, and that's where `f(a)?.foo` really shines. -- --Guido van Rossum (python.org/~guido)
On Sun, Sep 11, 2016 at 6:11 PM, Guido van Rossum <guido@python.org> wrote:
The non-repetition is certain a plus, I readily confess. It's not only performance even; `f()` might not be a pure function. Silly example:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
I messed up my answer and replied to one person instead of the list, so I'll post it again. There is also an alternative to this operator, and it's allowing a shortcut to do: try: val = do_thing() except ThingError: val = "default" In the form of: val = do_thing() except ThingError: "default" I was debated, and rejected, but I feel like mentioning it again because it has some strong benefits. First, it handles the null coalescing very quite well: val = obj.foo.bar.hey except AttributeError: "default" But it also can deal with many common operations in Python without the need to add more operators or variants: val = my_list[0] except IndexError: "default" val = iterable[0] except TypeError: next(iter(iterable)) val = int(param) except ValueError: man.nan It's quite readable, in the same vein of val = foo if bar else "default", but also familiar since it's using known keyword. And it doesn't require to add a new operator support in the parser. Another serious benefits is that it fits current use cases, AND futur use cases. Indeed, since it leverages Python exception mechanism, any lib implementing a clean error model can immediately let the users benefit from it without having to implement, tests and document lots of helper methods with "default" keywords and the like, while not being limited to a set of values some operators would only care about, such as None. Plus, since EAFP is a popular and handy pattern, it makes sense. At last, it has the same characteristic as the null coalescing operator: it's lazy, and hence has a small performance interest too compared to functional equivalent. Did I mention it's also easy to expend to a full try/except when you need something more complicated ? And then you benefit from else and finally immediately. I already have many code that would benefit from such a syntax, and I'd like to hear again what you think about it. Le 11/09/2016 à 21:44, Daniel Moisset a écrit :
On 09/12/2016 12:05 AM, Michel Desmoulin wrote:
+1 There are many places in my code where this would clean things up a bit. Not having it is like not having list comps or not having the ternary if-else -- possible, but it would be much nicer to have it. -- ~Ethan~
On 12 September 2016 at 09:05, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
I like this idea, I would propose a (maybe crazy) addition to it. What about a special exception NoneError, that will catch TypeError, AttributeError etc. but only when it was caused by None(), None.attr, None[1], etc. With this one can write: x = a.b()[0] except NoneError: 'default' without a risk of catching other (unrelated) exceptions. -- Ivan
On 12/09/2016 11:01, Ivan Levkivskyi wrote:
Assuming you can't break existing code that already traps TypeError, AttributeError, etc., I don't see how you can do this without having separated kinds of NoneError which were subclasses of TypeError, AttributeError, etc. Rob Cliffe
On Tue, Sep 13, 2016 at 12:03 AM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
class NoneError(Exception): pass class TypeNoneError(TypeError, NoneError): pass class AttributeNoneError(AttributeError, NoneError): pass Now you can catch NoneError to catch None.xyz, or AttributeError to catch foo.xyz for any foo. I don't think it's good, but it's possible. ChrisA
For the record, I still really don't like PEP 463. We should strive to catch fewer exceptions, not make it easier to catch them. On Mon, Sep 12, 2016 at 7:09 AM, Chris Angelico <rosuav@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)
On 12/09/2016 16:37, Guido van Rossum wrote:
Also I don't see the connection between the first sentence and the second. Exception-catching expressions as per PEP 463 just give you a more concise way of of doing something you can do with try+except. Do you mean it would make it easier to catch exceptions because you have to type fewer characters, and that this would be a bad thing? Thanks, Rob Cliffe
On Mon, Sep 12, 2016 at 5:45 PM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
(If that means "Leap Before You Look", yes. :-)
Yeah, that's exactly my point. PEP 463 gives you a shorter way to catch an exception, so it gives you less motivation to find a way to write your code (or define your API) that doesn't involve catching exceptions. But APIs involving exceptions are often inferior to APIs that don't require exception catching. (Yes, I am aware of __next__() raising StopIteration -- but that API design usually doesn't require you to catch it.) -- --Guido van Rossum (python.org/~guido)
On 13/09/2016 04:43, Guido van Rossum wrote:
You surprise me. I thought LBYL and EAFP were both approved Python idioms, in some cases one being better, in some cases another, choice to be made on the merits of each case (or the author's preference). I certainly use both (and sometimes time both to see which is faster). Now it sounds as if you're trying to impose a style guide on the world by discouraging the EAFP. And wasn't the discussion general, not about APIs specifically? Best wishes, Rob Cliffe
On 13 September 2016 at 21:15, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
Which is preferable depends greatly on context of use, which is why you'll find a lot of Python APIs offer both forms - it's not *just* a matter of inheriting the exceptionless version from C, and then later adding a Python version that gives an exception instead of None or a user-supplied default value. It's similar to why IEEE754 defines both quiet NaN *and* signalling NaN - which one you want depends on what you're doing. In web servers, for example, you'll often have lots of fields where "not present" is a perfectly acceptable return value. For those, APIs that just return None for unknown entries are very handy, which is why SQL Alchemy offers both ".first()" and "one()", which mainly differ in how and when they throw an exception, rather than what they do when they succeed. However, blindly catching *all* exceptions from a complex subexpression is rarely the right thing to do, so APIs that only offer "this may throw exceptions during normal operation under these circumstances" without a convenience wrapper that does the exception handling for you can end up being a pain to work with. PEP 463 makes those APIs less painful to deal with, but at the cost of encouraging overly broad exception handlers. By contrast, fixing APIs on a case-by-case basis puts the design burden where it can do the most good: on the API designer, who can make the scope of the exception handling suitably narrow *inside* the API implementation, rather than being limited to wrapping the entire API call in try/except. Cheers, Nick. P.S. There are also some use cases where Look-Before-You-Leap is inherently subject to race conditions, and for those, exceptions are the only reliable signaling mechanism. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I doubt very much it will be used for very complexe cases. Just like comprehensions or ternary expressions, they are a good fit for some specific use cases, and people will quickly catch on which one. You rarely see nested comprehensions or ternary expressions while it's possible to do so, because the Python communality values tend to limit abuses. It will be the same for this. You won't see very complex usages, mostly things like: val = foo[-1] except IndexError: "bar" doh = val.attr.other except AttributeError: "default" Those would already be coded with something similar (or maybe a chain of getattr()) or next(iter()). It's not a huge risk. But it's a huge convenience. Le 13/09/2016 à 13:37, Nick Coghlan a écrit :
On Tue, Sep 13, 2016, at 08:44, Michel Desmoulin wrote:
It occurs to me that if lambda were more lightweight [whether it's the syntax or the implementation that's the problem is unclear], not only this but also the ternary operator could have been functions. val = iexcept(lambda: foo[-1], IndexError, lambda: "bar")
On Tue, Sep 13, 2016, at 07:37, Nick Coghlan wrote:
P.S. There are also some use cases where Look-Before-You-Leap is inherently subject to race conditions,
Which use cases *aren't*?
and for those, exceptions are the only reliable signaling mechanism.
It's entirely possible to design a non-exception interface without race conditions. dict.get is *almost* a suitable example of such a method. Remember, at the C level in CPython, throwing an exception *is* returning a value [typically (PyObject*)NULL for interfaces that otherwise return an object], along with setting the error indicator. The only missing piece is an "error value" outside the set of valid values. Guido's argument here seems to be that exception-based EAFP is not pythonic.
On Wed, Sep 14, 2016 at 12:14 AM, Random832 <random832@fastmail.com> wrote:
Ones in which no external force can affect things. For example: def frob(spam): if spam is None: print("Frobbing nothing") else: print("Frobbing some spam: ", spam) ... You can safely assume that locals won't be changed between the 'is None' check and the print. I suppose someone could mess around with thread call stacks, but that'd be seriously insane. ChrisA
On 13/09/2016 12:37, Nick Coghlan wrote:
"Guido's argument here seems to be that exception-based EAFP is not pythonic." [snip] try+except block. Of course result = (myList[0] except Exception: MyDefault) is poor code (presumably it should have been written "except IndexError"), but so is try: result = myList[0] except Exception: result = MyDefault ISTM you're giving an exception-catching dog a bad name and hanging him. Or have I missed something? And sorry to repeat myself, but we seemed to be having a *general* discussion about null-coalescing operators, which moved on to PEP 463, then suddenly Guido is talking about APIs. A lot of the time when I'm coding, I'm not writing an API, just a program to get a job done. No doubt, exceptions should be discouraged *in APIs*, but that doesn't make exceptions, or EAFP, a bad idea per se. I really don't mean this post to sound hostile, and I'm sorry if it comes across a bit that way. I'm just surprised at what I'm hearing, and to be honest, I have a soft spot for PEP 463. Best wishes Rob Cliffe
On Wed, Sep 14, 2016 at 1:27 AM, Rob Cliffe <rob.cliffe@btinternet.com> wrote:
PEP 463, by its nature, is talking about the language. Language features govern and advise API design, which then changes how you write "a program to get a job done", so it does actually have strong bearing - just via a couple of levels of indirection. ChrisA
On 2016-09-13 16:27, Rob Cliffe wrote:
[snip] I think the point is that exceptions are for, well, exceptional processing, not normal processing. Code should be written on the assumption that everything works normally, e.g. division always returns a result. _Occasionally_ division will fail, such as attempting to divide by zero, but that should be uncommon. If something pretty much always succeeds, use an exception for failure, but if it sometimes succeeds and sometimes fails, check explicitly for success or failure.
This general API discussion is veering off-topic for the subject line. If people would like to continue to discuss it then please start a new thread (if you feel it impacts your view of a null-coalescing operator then please discuss in the new thread and come back to this one). On Tue, 13 Sep 2016 at 11:25 MRAB <python@mrabarnett.plus.com> wrote:
On 09/12/2016 08:37 AM, Guido van Rossum wrote:
For the record, I still really don't like PEP 463. We should strive to catch fewer exceptions, not make it easier to catch them.
I certainly agree with the first part, slightly reworded: we should strive to generate fewer exceptions that we have to catch. I disagree with the second part: being able to condense four lines of code (1 for try, 1 for except, 1 for the attempt, and 1 for recovery) in to one line of code seems like a win. I know I find it frustrating when my choice is between the 4-line boiler-plate try/except, or an equally verbose and ugly multi-line non-exception generating stanza. Anyway, my two cents worth. If you collect them all for this subject I think I owe you a dime. ;) -- ~Ethan~
On 12/09/2016 08:05, Michel Desmoulin wrote:
+1, you're preaching to the converted. This is PEP 463, "Exception-catching expressions". Except that the PEP required parentheses around a whole exception-catching expression, for reasons that are not clear to me, i.e. val = (my_list[0] except IndexError: "default") rather than val = my_list[0] except IndexError: "default" Rob Cliffe
On Mon, Sep 12, 2016 at 1:05 AM, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
Note that there's a subtle difference here when multiple lookups are involved. Given: def f(spam): return spam().eggs().ham With null-coalescing: def f(spam): return spam()?.eggs()?.ham This is roughly equivalent to: def f(spam): _spam = spam() try: eggs = _spam.eggs except AttributeError: return None _eggs = eggs() try: return _eggs.ham except AttributeError: return None With PEP 463 it doesn't work out so well. The "obvious" spelling would be: def f(spam): return (spam().eggs().ham except AttributeError: None) This is roughly equivalent to: def f(spam): try: return spam().eggs().ham except AttributeError: return None Note how it's different. For one thing, it could mask AttributeError coming from the calls. For another, you no longer explicitly identify which lookups to handle. I would expect both to lead to subtle bugs, whereas with null-coalescing you don't have those problems. -eric
On 12 September 2016 at 21:47, Eric Snow <ericsnowcurrently@gmail.com> wrote:
From previous explanations in this thread, that's *not* the behaviour at all.
If I understand the proposal, f is actually intended to be equivalent to: def f(spam): spam_val = spam() if spam_val is None: return None eggs_val = spam_val.eggs() if eggs_val is None: return None return eggs_val.ham Personally, I find it pretty worrying that there's this persistent confusion over what the proposed syntax means. Sure, it's explained in the PEP and proposal, but if it gets implemented it'll be in the docs. And yet people don't seem to read any of those - and their intuition of what the construct does is wrong. IMO, the syntax may well be useful, but if we don't address the problem that what people *think* it means isn't what it actually means, then it'll do more harm than good. Paul
On 09/12/2016 02:13 PM, Paul Moore wrote:
On 12 September 2016 at 21:47, Eric Snow wrote:
Perhaps the bumper-sticker explanation is: None-coalescing is not to /recover/ from an AttributeError, but to _prevent it_ in the first place -- but only when the base object is already None. Okay, several bumper stickers. ;) -- ~Ethan~
On Mon, Sep 12, 2016 at 3:13 PM, Paul Moore <p.f.moore@gmail.com> wrote:
From previous explanations in this thread, that's *not* the behaviour at all.
Gah, you're right. I knew better too. The diversion back to PEP 463 crossed some wires in my brain. :)
...which is even less related to PEP 463.
The tricky bit is the syntactic association with attribute lookup. However, I don't think it's quite so confusing. As noted above, my confusion was mostly due to the recent mention of PEP 463. -eric
On 13 September 2016 at 07:13, Paul Moore <p.f.moore@gmail.com> wrote:
Right, there are two quite reasonable interpretations for what a conditional attribute lookup does, and the "catch and ignore AttributeError" case is going to be more familiar to most Pythonistas, as that's the way getattr() works when given a default value. Consider the case of chained attribute lookup as a function using a "None" default to getattr(): def attr_chain_abort_on_missing(base, *attrs): result = base for attr in attrs: if result is None: break result = getattr(result, attr, None) return result vs the actually proposed semantics: def attr_chain_abort_only on_none(base, *attrs): result = base for attr in attrs: if result is None: break result = getattr(result, attr) return result In the latter version, AttibuteError escapes - it's only when the original value is None, or an attribute exists and is None that the iteration bails out early without raising an exception.
Agreed, and I think a key indicator for that would be whether or not people that saw: result = obj?.first?.second?.third agreed on whether or not it could raise AttributeError or TypeError. Any prospective PhD students in the audience looking for a language usability question to study before 3.7b1 rolls around in 2018? :) Cheers, Nick. P.S. I'll note that the *upside* I see to (this part of) the proposal is that it would implement a *very* fine-grained "is not None" check, where the alternative in real code would often be an overly broad exception handling clause, like: try: result = obj.first().second.third except (TypeError, AttributeError): return None rather then actually spelling out the fine-grained checks properly. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sat, Sep 10, 2016, at 20:15, David Mertz wrote:
To put it more explicitly, more similar to "a and a.b"
Why? I think you're confusing this proposal for something else. This is for treating x is None specially, not simply catching AttributeError to deal with the fact that None.foo doesn't exist.
On Sat, Sep 10, 2016, at 19:38, David Mertz wrote:
x2 = x?.foo
x3 = x?.bar?.baz?[x2]
A. if you're doing three different things with x, why are you using this instead of wrapping it in an if statement? B. Under some of the proposals, unless x.bar might be None independently of x not being None, this would just be "x?.bar.baz[x2]
x4 = x?(x?.val)
C. Under some of the proposals, the inner contents of the call brackets aren't evaluated if x is None, so you don't need to use x?.val here.
result = x3?(x4)
participants (15)
-
Brett Cannon -
Bruce Leban -
Chris Angelico -
Daniel Moisset -
David Mertz -
Eric Snow -
Ethan Furman -
Guido van Rossum -
Ivan Levkivskyi -
Michel Desmoulin -
MRAB -
Nick Coghlan -
Paul Moore -
Random832 -
Rob Cliffe