[Python-ideas] PEP 530: Asynchronous Comprehensions

[Nick Coghlan]

On 7 September 2016 at 21:37, Andrew Svetlov <andrew.svetlov at gmail.com> wrote:
> On Wed, Sep 7, 2016 at 2:31 PM Nick Coghlan <ncoghlan at gmail.com> wrote:
>> After using it a few times in examples, while I'm prepared to accept
>> the agrammatical nature of "async for" in the statement form (where
>> the adjective-noun phrase can be read as a kind of compound noun
>> introducing the whole statement), I think for the comprehension form,
>> we should aim to put the words in the right grammatical order if we
>> can:
>>
>>     result = [i for i in async aiter() if i % 2]
>>
> Please, no.
> It may be totally correct from English grammar POV but brings different
> syntax from regular async for statement, e.g.
> async for row in db.execute(...):
>     pass

The new issue that's specific to comprehensions is that the statement
form doesn't have the confounding factor of having an expression to
the left of it. Thus, the prefix is unambiguous and can be read as
modifying the entire statement rather than just the immediately
following keyword:

    async for row in db.execute(...):
        process(row)

It's also pragmatically necessary right now due to the sleight of hand
that Yury used in the code generation pipeline to bring in "async
def", "async with", and "async for" without a __future__ statement,
but without making them full keywords either.

However, when we convert it to the comprehension form, a parsing
ambiguity (for humans) arises that creates an inherent readability
problem:

    [process(row) async for row in db.execute(...)]

When reading that, is "async" a postfix operator being used in a
normal comprehension (wrong, but somewhat plausible)? Or is it part of
a compound keyword with "for" that modifies the iteration behaviour of
that part of the comprehension (the correct interpretation)?

    [(process(row) async) for row in db.execute(...)]
    [process(row) (async for) row in db.execute(...)]

The postfix operator interpretation is just plain wrong, but even the
correct interpretation as a compound keyword sits between two
expressions *neither* of which is the one being modified (that would
be "db.execute()")

By contrast, if the addition of full async comprehensions is deferred
to 3.7 (when async becomes a true keyword), then the infix spelling
can be permitted in both the statement and comprehension forms:

    for row in async db.execute(...):
        process(row)

    [process(row) for row in async db.execute(...)]

with the prefix spelling of the statement form retained solely for
backwards compatibility purposes (just as we retain "from __future__
import feature" flags even after the feature has become the default
behaviour).

The beauty of the infix form is that it *doesn't matter* whether
someone reads it as a compound keyword with "in" or as a prefix
modifying the following expression:

    [process(row) for row (in async) db.execute(...)]
    [process(row) for row in (async db.execute(...))]

In both cases, it clearly suggests something special about the way
"db.execute()" is going to be handled, which is the correct
interpretation.

> Currently regular comprehensions are pretty similar to `for` loop.
> Why async comprehensions should look different from `async for` counterpart?

Regular "for" loops don't have the problem of their introductory
keyword being written as two words, as that's the culprit that creates
the ambiguity when you add an expression to the left of it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia