[Python-ideas] Coming up with an alternative to PEP 505's None-aware operators

Nick Coghlan ncoghlan at gmail.com
Thu Feb 15 21:06:04 EST 2018

The recent thread on variable assignment in comprehensions has
prompted me to finally share
https://gist.github.com/ncoghlan/a1b0482fc1ee3c3a11fc7ae64833a315 with
a wider audience (see the comments there for some notes on iterations
I've already been through on the idea).

== The general idea ==

The general idea would be to introduce a *single* statement local
reference using a new keyword with a symbolic prefix: "?it"

* `(?it=expr)` is a new atomic expression for an "it reference
binding" (whitespace would be permitted around "?it" and "=", but PEP
8 would recommend against it in general)
* subsequent subexpressions (in execution order) can reference the
bound subexpression using `?it` (an "it reference")
* `?it` is reset between statements, including before entering the
suite within a compound statement (if you want a persistent binding,
use a named variable)
* for conditional expressions, put the reference binding in the
conditional, as that gets executed first
* to avoid ambiguity, especially in function calls (where it could be
confused with keyword argument syntax), the parentheses around
reference bindings are always required
* unlike regular variables, you can't close over statement local
references (the nested scope will get an UnboundLocalError if you try

The core inspiration here is English pronouns (hence the choice of
keyword): we don't generally define arbitrary terms in the middle of
sentences, but we *do* use pronouns to refer back to concepts
introduced earlier in the sentence. And while it's not an especially
common practice, pronouns are sometimes even used in a sentence
*before* the concept they refer to ;)

If we did pursue this, then PEPs 505, 532, and 535 would all be
withdrawn or rejected (with the direction being to use an it-reference

== Examples ==

`None`-aware attribute access:

    value = ?it.strip()[4:].upper() if (?it=var1) is not None else None

`None`-aware subscript access:

    value = ?it[4:].upper() if (?it=var1) is not None else None


    value = ?it if (?it=var1) is not None else ?it if (?it=var2) is
not None else var3


    value = ?it if not math.isnan((?it=var1)) else ?it if not
math.isnan((?that=var2)) else var3

Conditional function call:

    value = ?it() if (?it=calculate) is not None else default

Avoiding repeated evaluation of a comprehension filter condition:

    filtered_values = [?it for x in keys if (?it=get_value(x)) is not None]

Avoiding repeated evaluation for range and slice bounds:

    range((?it=calculate_start()), ?it+10)

Avoiding repeated evaluation in chained comparisons:

    value if (?it=lower_bound()) <= value < ?it+tolerance else 0

Avoiding repeated evaluation in an f-string:

    print(f"{?it=get_value()!r} is printed in pure ASCII as {?it!a}
and in Unicode as {?it}"

== Possible future extensions ==

One possible future extension would be to pursue PEP 3150, treating
the nested namespace as an it reference binding, giving:

    sorted_data = sorted(data, key=?it.sort_key) given ?it=:
        def sort_key(item):
            return item.attr1, item.attr2

(A potential bonus of that spelling is that it may be possible to make
"given ?it=:" the syntactic keyword introducing the suite, allowing
"given" itself to continue to be used as a variable name)

Another possible extension would be to combine it references with `as`
clauses on if statements and while loops:

    if (?it=pattern.match(data)) is not None as matched:

    while (?it=pattern.match(data)) is not None as matched:

== Why not arbitrary embedded assignments? ==

Primarily because embedded assignments are inherently hard to read,
especially in long expressions. Restricting things to one pronoun, and
then pursuing PEP 3150's given clause in order to expand to multiple
statement local names should help nudge folks towards breaking things
up into multiple statements rather than writing ever more complex

That said, the ?-prefix notation is deliberately designed such that it
*could* be used with arbitrary identifiers rather then being limited
to a single specific keyword, and the explicit lack of closure support
means that there wouldn't be any complex nested scope issues
associated with lambda expressions, generator expressions, or
container comprehensions.

With that approach, "?it" would just be an idiomatic default name like
"self" or "cls" rather than being a true keyword. Given arbitrary
identifier support, some of the earlier examples might instead be
written as:

    value = ?f() if (?f=calculate) is not None else default
    range((?start=calculate_start()), ?start+10)
    value if (?lower=lower_bound()) <= value < ?lower+tolerance else 0

The main practical downside to this approach is that *all* the
semantic weight ends up resting on the symbolic "?" prefix, which
makes it very difficult to look up as a new Python user. With a
keyword embedded in the construct, there's a higher chance that folks
will be able to guess the right term to search for (i.e. "python it
expression" or "python it keyword").

Another downside of this more flexible option is that it likely
*wouldn't* be amenable to the "if expr as name:" syntax extension, as
there wouldn't be a single defined pronoun expression to bind the name

However, the extension to PEP 3150 would allow the statement local
namespace to be given an arbitrary name:

    sorted_data = sorted(data, key=?ns.sort_key) given ?ns=:
        def sort_key(item):
            return item.attr1, item.attr2


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Python-ideas mailing list