[Python-Dev] PEP 492 vs. PEP 3152, new round

[Łukasz Langa]

> On Apr 24, 2015, at 10:03 AM, Guido van Rossum <guido at python.org> wrote:
> 
> 1. precise syntax of `async def`
> 
> So I still prefer `async def`.

Me too. Also, we would use a similar vocabulary to existing users of the feature. This is exactly how Hack does it: http://docs.hhvm.com/manual/en/hack.async.php <http://docs.hhvm.com/manual/en/hack.async.php>, how ECMAScript 7 proposes it: http://wiki.ecmascript.org/doku.php?id=strawman:async_functions <http://wiki.ecmascript.org/doku.php?id=strawman:async_functions> and similarly to how C# does it (async comes after the public/private modifier but before the return type): https://msdn.microsoft.com/en-us/library/hh156513.aspx <https://msdn.microsoft.com/en-us/library/hh156513.aspx>


> 2. do we need `async for` and `async with`
> 
> Yes we do.

+1


> (Though maybe we should consider `await for` and `await with`? That would have the advantage of making it easy to scan for all suspension points by searching for /await/. But being a verb it doesn't read very well.)

I’m on the fence here.

OT1H, I think “await for something in a_container” and “await with a_context_manager():” also read pretty well. It’s also more consistent with being the one way of finding “yield points”.

OTOH, “await with a_context_manager():” suggests the entire statement is awaited on, which is not true. “async with” is more opaque in this way, it simply states “there are going to be implementation-specific awaits inside”. So it’s more consistent with “async def foo()”.

All in all I think I’m leaning towards “async for” and “async with”. More importantly though, I’m wondering how obvious will the failure mode be when somebody uses a bare “for” instead of an “async for”. Ditto for “with” vs. “async with”. How much debugging will be necessary to find that it’s only a missing “async” before the loop?


Side note: to add to the confusion about syntax, Hack’s equivalent for “async for” - which doesn’t really translate well to Python - uses “await” and ties it to the iterable:

foreach ($list await as $input) { … }

The equivalent in Python would be:

for input in list await:

Notably, this is ugly and would be confused with `for input in await list:` which means something different.

Also, this particular construct represents less than 0.01% of all “await” occurences in Facebook code, suggesting it’s not performance critical.


> 3. syntactic priority of `await`
> 
> Yury, could you tweak the syntax for `await` so that we can write the most common usages without parentheses?

+1

Yury points out there was likely a reason this wasn’t the case for `yield` in the first place. It would be good to revisit that. Maybe for yield itself, too?


> 4. `cocall` vs. `await`
> 
> I just can't get used to this aspect of PEP 3152, so I'm rejecting it.

+1


> (Yury: PEP 492 is not accepted yet, but you're getting closer.)

May I suggest using the bat-signal to summon Glyph to confirm this is going to be helpful/usable with Twisted as well?


> 5. do we really need `__aiter__` and friends
> 
> There's a lot of added complexity, but I think it's worth it. I don't think we need to make the names longer, the 'a' prefix is fine for these methods.

+1


> 6. StopAsyncException
> 
> I'm not sure about this. The motivation given in the PEP seems to focus on the need for `__anext__` to be async. But is this really the right pattern? What if we required `ait.__anext__()` to return a future, which can either raise good old `StopIteration` or return the next value from the iteration when awaited? I'm wondering if there are a few alternatives to be explored around the async iterator protocol still.

So are you suggesting to pass the returned value in a future? In this case the future would need to be passed to __anext__, so the Cursor example from the PEP would look like this:

class Cursor:
    def __init__(self):
        self.buffer = collections.deque()

    def _prefetch(self):
        ...

    async def __aiter__(self):
        return self

    async def __anext__(self, fut):
        if not self.buffer:
            self.buffer = await self._prefetch()
        if self.buffer:
            fut.set_result(self.buffer.popleft())
        else:
            fut.set_exception(StopIteration)

While this is elegant, my concern is that one-future-per-iteration-step might be bad for performance.

Maybe consider the following. The `async def` syntax decouples the concept of a coroutine from the implementation. While it’s still based on generators under the hood, the user no longer considers his “async function” to be a generator or conforming to the generator protocol. From the user’s perpective, it’s obvious that the return below means something different than the exception:

    async def __anext__(self):
        if not self.buffer:
            self.buffer = await self._prefetch()
            if not self.buffer:
                raise StopIteration
        return self.buffer.popleft()

So, the same way we added wrapping in RuntimeErrors for generators in PEP 479, we might add transparent wrapping in a _StopAsyncIteration for CO_COROUTINE.

> 7. compatibility with asyncio and existing users of it

+1, this is really important.

-- 
Best regards,
Łukasz Langa

WWW: http://lukasz.langa.pl/
Twitter: @llanga
IRC: ambv on #python-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150424/00c46055/attachment.html>