[Python-Dev] Alternative forms [was: PEP 463: Exception-catching expressions]

[Chris Angelico]

On Sat, Mar 8, 2014 at 5:58 AM, Jim J. Jewett <jimjjewett at gmail.com> wrote:
>
> (Thu Mar 6 23:26:47 CET 2014) Chris Angelico responded:
>
>> On Fri, Mar 7, 2014 at 7:29 AM, Jim J. Jewett <jimjjewett at gmail.com> wrote:
>
>>> [ note that "x if y" already occurs in multiple contexts, and
>>>   always evaluates y before x. ]
>
>> Yes, but that's still out of order.
>
> Yeah, but local consistency is more important than global guidelines.  :D

I don't see except expressions as fundamentally more associated with
if/else than with, say, an or chain, which works left to right. Aside
from both being ternary operators, of course, which isn't really much
of a justification.

>> The other thing to note is that it's somewhat ambiguous. Until you
>> find that there isn't an else clause, it could be the equally valid
>> "expr except (default if cond else other_default)", with the actual
>> "if Exception" part still to come.
>
> True -- and I'm not a parser expert.  But my belief is that the
> current parser allows lookahead for exactly one token, and that
> the "else" would fit within that limit.
>
>> ... humans reading the code have to assume style guides mightn't
>> be followed.
>
> True ... but I hope any non-trivial use of this (including use
> with a non-trivial ternary if) will look bad enough to serve as
> its own warning.

Not sure whether the parser would be able to handle it or not, but the
human would have to if the machine can, and that's going to be a
source of confusion. I'd rather avoid it if I can. Remember,
everywhere else in Python, the word "if" is followed by something
that's interpreted as a boolean. You wouldn't expect to see this
somewhere:

if None:
    do_stuff()

do_stuff() if sys else do_other_stuff()

So it'd cause a mental glitch to see some other constant and
always-true expression there:

... if ZeroDivisionError ...

You don't expect ZeroDivisionError ever to be false. Seeing it
following an if would leave you wondering if maybe it is.

IMO that syntax is abuse of the 'if' keyword, hammering it into a hole
it's not designed to fit into.

> I do think parentheses help, (but are less important when there
> is only a single "if")

Analysis of the Python standard library suggests that the single-if
situation is *by far* the most common, to the extent that it'd hardly
impact the stdlib at all to add multiple except clauses to the
proposal. Do you have a strong use-case for the more full syntax? It
tends to get long, which rather damages the readability. A number of
people have said that if the except expression goes over a line break,
it should probably be a statement instead. I'm not sure that extreme
is fully justified, but it certainly does have merit. In a codebase of
mine I wouldn't *forbid* breaking an except expression over multiple
lines, but it'd certainly be a reason to consider whether it's really
necessary or not.

> and I strongly prefer that they [the parentheses] be internal
> (which you fear looks too much like calling a function named except).
> In that case, it is:
>
>     expr1 except (expr3 if expr2)

I'm still not really seeing how this is better. With the colon
version, it looks very much like dict display, only with different
brackets around it; in some fonts, that'll be very easily confused.
With the if, it looks like an incomplete expression waiting for its
else. And I'm still not enamored of constructing syntax that has the
evaluation order (a) not simple left-to-right, like most expressions
are, and (b) different from the except statement, which puts the
exception_list ahead of the suite.

> Agreed ... the "say it like you would in English" applies only
> to the "expr if expr" form (proposed here and) used by comprehensions:
>
>     [1/x for x in data
>          if x]

Sure. That usage makes a _lot_ of sense, I really like list
comprehensions. There's no room in them for an 'else' clause though,
so it's very strictly one-way. Makes them a tricky comparison for
this.

> I can't speak to the lambda precedent, but I do know that I personally
> often stumble when trying to parse it, so I don't consider it a good model.

The only confusion I have with lambda is its precedence, which trips
me up now and then:

>>> def f(x):
...     return lambda y: x,y
...
>>> f(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f
NameError: name 'y' is not defined

That's not returning a function that returns a tuple. It's returning a
tuple of a function and the global y, and the function will ignore its
arg.

> The other three inline uses (dict display, slide notation, and
> function parameter annotation) are effectively conjunction operators,
> saying that expr1 and expr2 are bound more tightly than you would
> assume if they were separated by commas.  They only occur inside
> a fairly close bracket (of some sort), and if the bracket isn't
> *very* close, then there are usually multiple associates-colons
> inside the same bracket.

Not sure that really helps. This isn't going to be as tight as a
slice, and it's most often not going to have multiple colons inside
the brackets.

> Without (preferably internal) parentheses, it will instead look like
> a long line with a colon near the end, and a short continuation suite
> that got moved up a line because it was only one statement long.
>
>     def nullfunc(self, a): pass
>
>     expr1 except expr3: expr2

That should have expr2 and exp3 the other way round - the bit before
the colon is evaluated before the bit after the colon.

What about putting it all in parens, which is the current proposal?
There's no way you can have a statement with a one-line suite on the
same line, all enclosed in parens.

(expr1 except expr2: expr3)

>>>> All forms involving the 'as' capturing clause have been deferred ...
>
>>> Nick is right that you should specify whether it is deferred or
>>> rejected, because the simplest implementation may lock you into
>>> too broad a scope if it is added later.
>
>> Put it this way: It's deferred until there's a non-closure means of
>> creating a sub-scope.
>
> The problem is that once it is deployed as leaking into the parent
> scope, backwards compatibility may force it to always leak into
> the parent scope.  (You could document the leakage as a bug or as
> implementation-defined, but ... those choices are also sub-optimal.)

It'll never be deployed as leaking, for the same reason that the
current 'except' statement doesn't leak: it creates a refloop. The
entire notion of an 'as' clause is deferred. If there's no way to
create a sub-scope that isn't a closure, then there will be no 'as'
clause in an except expression. It's that simple. Then later, if such
a feature is added, it can be retrofitted without breaking anything.
But there's little need of it; the cases where the exception needs to
be captured are usually the ones that need the full statement anyway.
Of all 'except' statements in the stdlib, almost exactly one in five
uses 'as'; of the ones simple enough to potentially be changed to this
form, just 7 out of 236 do. (And those seven are among the more
dubious ones.) So it's no great loss to the proposal, assuming the
stdlib is indicative. (If you have a good-sized Py3 codebase to try it
on, feel free to enhance those stats with more data. The script's part
of my repo.)

> The fact that short-circuiting will prevent false_expr from even
> being evaluated means that "(true_expr if cond)" is a useful
> mental approximation, even though "true_expr if cond" is already
> to the left.

Hrm, I don't really like that way of describing it. To me, the entire
expression is one unit - you can't parenthesize it. It's like
parenthesizing a chained comparison:

if 5 < x < 10:
if (5 < x) < 10:
if 5 < (x < 10):

You can't add parens around one part of the expression; it completely
changes the meaning of it.

> Commutivity and Associativity are pretty strong biases; testing
> order of execution for apparently parallel expressions (such as
> different arguments to the same function in the same call) is a
> source of trick questions.

In C, the answer is usually "Implementation-defined". In Python, the
answer is usually "Left to right".

> The replacement value and the exception that triggers it are
> clearly associated more closely with each other than either is
> to the primary expression.  Therefore, they *should* be grouped
> more tightly.

Maybe. On the other hand, the replacement value and the original
expression are two things of the same broad type, and the other
expression is an exception list, so the primary and the replacement
should be grouped more tightly. You're going to get this or that. I
don't think this is a strong reason for parentheses.

> This is a proposal with no perfect syntax, but I don't really
> see any upside to *not* grouping the exception with its
> exceptional result.  At the moment, I don't know how to spell
> that closer grouping without at least one of:
>
>     Parentheses or brackets of some sort
>
>     A line break/new suite:
>         - kills the proposal
>
>     An inline associates-colon:
>         - needs brackets anyhow, or it will be mistaken for
>           a suite-introduction-colon.
>
>     A new grouping operator, such as "->"
>         - risk that it will be misinterpreted
>
>     Relying on convention and taste:
>         (main_expr          except exc_expr fallback_expr)

Definitely not the last one. It needs some kind of syntactic
separation, not just whitespace (what if the fallback_expr is a
negative number?). The entire expression should be representing a
single thought, or else it probably should be written as a statement.
Like the thought "Get me the current working directory, None if there
isn't one".

pwd = (os.getcwd() except OSError: None)

Or: "Append the IP, or failing that the network address".

ips.append(ip.ip except AttributeError: ip.network_address)

With those, there's really no separation between "Get the current
working directory", "if there isn't one", and "None". Parenthesizing
the part after 'except' makes good sense if you want to have multiple
except clauses, but there's no reason to do that, so we're back with
this very simple case.

I'd rather not mandate any parens at all, personally, but as was said
in another thread, you don't need to see a Frenchman on the parapet to
know where the wind's blowing.

ChrisA