PEP 492: __aiter__ should return async iterator directly instead of awaitable
There is a small flaw in PEP 492 design -- __aiter__ should not return an awaitable object that resolves to an asynchronous iterator. It should return an asynchronous iterator directly. Let me explain this by showing some examples. I've discovered this while working on a new asynchronous generators PEP. Let's pretend that we have them already: if we have a 'yield' expression in an 'async def' function, the function becomes an "asynchronous generator function": async def foo(): await bar() yield 1 await baz() yield 2 # foo -- is an `asynchronous generator function` # foo() -- is an `asynchronous generator` If we iterate through "foo()", it will await on "bar()", yield "1", await on "baz()", and yield "2": >>> async for el in foo(): ... print(el) 1 2 If we decide to have a class with an __aiter__ that is an async generator, we'd write something like this: class Foo: async def __aiter__(self): await bar() yield 1 await baz() yield 2 However, with the current PEP 492 design, the above code would be invalid! The interpreter expects __aiter__ to return a coroutine, not an async generator. I'm still working on the PEP for async generators, targeting CPython 3.6. And once it is ready, it might still be rejected or deferred. But in any case, this PEP 492 flaw has to be fixed now, in 3.5.2 (since PEP 492 is provisional). I've created an issue on the bug tracker: http://bugs.python.org/issue27243 The proposed patch fixes the __aiter__ in a backwards compatible way: 1. ceval/GET_AITER opcode calls the __aiter__ method. 2. If the returned object has an '__anext__' method, GET_AITER silently wraps it in an awaitable, which is equivalent to the following coroutine: async def wrapper(aiter_result): return aiter_result 3. If the returned object does not have an '__anext__' method, a DeprecationWarning is raised.
On Jun 6, 2016, at 12:23 PM, Yury Selivanov
However, with the current PEP 492 design, the above code would be invalid! The interpreter expects __aiter__ to return a coroutine, not an async generator.
Yes, I remember asking about the reason behind __aiter__ being an awaitable during the original PEP 492 review process. You added an explanation to the PEP but I don’t think we ever had an example where this was needed. I’m +1 to resolve this now.
The proposed patch fixes the __aiter__ in a backwards compatible way:
1. ceval/GET_AITER opcode calls the __aiter__ method.
2. If the returned object has an '__anext__' method, GET_AITER silently wraps it in an awaitable, which is equivalent to the following coroutine:
async def wrapper(aiter_result): return aiter_result
3. If the returned object does not have an '__anext__' method, a DeprecationWarning is raised.
There’s a problem with this approach. It will force people to write deprecated code because you never know if your library is going to run on 3.5.0 or 3.5.1. Barry, Ubuntu wily, xenial and yakkety currently package 3.5.0 or 3.5.1. When 3.5.2 is going to get released, are they going to get it? I’m pretty sure wily isn’t and yakkety is but just wanted to confirm; especially with xenial being an LTS release. -- Not-that-i-see-a-different-way-out’sly yours, Ł
On 2016-06-06 4:02 PM, Łukasz Langa wrote:
The proposed patch fixes the __aiter__ in a backwards compatible way:
1. ceval/GET_AITER opcode calls the __aiter__ method.
2. If the returned object has an '__anext__' method, GET_AITER silently wraps it in an awaitable, which is equivalent to the following coroutine:
async def wrapper(aiter_result): return aiter_result
3. If the returned object does not have an '__anext__' method, a DeprecationWarning is raised.
There’s a problem with this approach. It will force people to write deprecated code because you never know if your library is going to run on 3.5.0 or 3.5.1. Barry, Ubuntu wily, xenial and yakkety currently package 3.5.0 or 3.5.1. When 3.5.2 is going to get released, are they going to get it? I’m pretty sure wily *isn’t* and yakkety *is* but just wanted to confirm; especially with xenial being an LTS release.
Yes, I agree. OTOH, I don't see any other way of resolving this. Another option would be to start raising the DeprecationWarning only in 3.6. Yury
The RC for 3.5.2 is going out coming weekend (see PEP 478
https://www.python.org/dev/peps/pep-0478/). We should get this out now,
or make it the first incompatibility in 3.6 (that's also an option; 3.6
feature freeze starts September, see PEP 494
https://www.python.org/dev/peps/pep-0494/).
On Mon, Jun 6, 2016 at 1:05 PM, Yury Selivanov
On 2016-06-06 4:02 PM, Łukasz Langa wrote:
The proposed patch fixes the __aiter__ in a backwards compatible way:
1. ceval/GET_AITER opcode calls the __aiter__ method.
2. If the returned object has an '__anext__' method, GET_AITER silently wraps it in an awaitable, which is equivalent to the following coroutine:
async def wrapper(aiter_result): return aiter_result
3. If the returned object does not have an '__anext__' method, a DeprecationWarning is raised.
There’s a problem with this approach. It will force people to write deprecated code because you never know if your library is going to run on 3.5.0 or 3.5.1. Barry, Ubuntu wily, xenial and yakkety currently package 3.5.0 or 3.5.1. When 3.5.2 is going to get released, are they going to get it? I’m pretty sure wily *isn’t* and yakkety *is* but just wanted to confirm; especially with xenial being an LTS release.
Yes, I agree. OTOH, I don't see any other way of resolving this.
Another option would be to start raising the DeprecationWarning only in 3.6.
Yury
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
On Jun 06, 2016, at 01:02 PM, Łukasz Langa wrote:
There’s a problem with this approach. It will force people to write deprecated code because you never know if your library is going to run on 3.5.0 or 3.5.1. Barry, Ubuntu wily, xenial and yakkety currently package 3.5.0 or 3.5.1. When 3.5.2 is going to get released, are they going to get it? I’m pretty sure wily isn’t and yakkety is but just wanted to confirm; especially with xenial being an LTS release.
Matthias and I talked briefly about this at Pycon. We want to get 3.5.2 into Ubuntu 16.04.1 if it's released in time. 16.04.1 is currently scheduled for July 21st [1] so if Larry keeps with his announced schedule that should work out[2]. Obviously it would make it into Yakkety too. It's not worth it for Wily (15.10) since that EOLs next month. Cheers, -Barry [1] https://wiki.ubuntu.com/XenialXerus/ReleaseSchedule [2] https://mail.python.org/pipermail/python-dev/2016-April/144383.html
participants (4)
-
Barry Warsaw -
Guido van Rossum -
Yury Selivanov -
Łukasz Langa